You can just leave your car on XM. The only issue with Android Auto is switching between a USB stick I have loaded with music and the music on my phone, after some trial and error the adapter or my car notices what I want for to run as my default.... Unless you are running XM off of your phone if could cause a minor inconvenience of changing your android auto preferences... so far this thing has been quite user friendly...
My only issues are if I am in a area with bad reception, the maps app tends to think you never left your destination...