Hey pynkneewyze,
Oh no, I'd be happy to help you with your problem. There is a little lock switch on top of where the adapter plugs in. You can slide the switch from the lock to the unlock position and slide the little adapter out. You will slide the adapter into the base and you should not have to use excessive force to slide it back out.
Since I see you have a Note 7, i would recommend participating in the US Product Exchange Program because Samsung cares about their customers safety and satisfaction. You can go to your place of purchase and exchange the phone for an S7, S7 Edge or a new Note 7, depending on availability. Please be aware that it is a one-time exchange and phone options are dependent on carrier availability. I hope you find this helpful and have a great day.