I tested that myself, today. Just before heading home from an appointment. I opened the Google Home App, found my devices list, and there were my lights. I turned them both on and set two different brightness levels. When we got home, both lights were on, and set to the different brightness levels. It works.
The Google Assistant device nears to be nearby to find the lights for setup. Unless your lights are within the 40 ft range (suggested by Google/GE) I don't think they can be controlled. That said, each Assistant can control a lot of lights, all with unique names (such as, Master 1, Master 2, Master 3, etc.).