No, the design of the bumper allows the band to be removed without issue as well as be charged without issue. You do not need to ever remove the bumper at all except if you decide to clean it. Hope this helps!
Case-Mate - Naked Tough Bumper for Apple Watch 38mm - Clear
(387)
Case-Mate - Naked Tough Bumper for Apple Watch 42mm - Clear
(517)