Hi Carrie,
Yes, after replacing the water filter, you need to dispense several gallons of water to clear out any impurities or air. The water pitcher does not have a filter; it only has a diffuser inside.
We shared a link below for guidelines on changing the water filter on the refrigerator. Please check the embedded words and watch the video for more details.
How to replace the water filter in a Samsung refrigerator
https://www.samsung.com/us/support/answer/ANS10005090/
Have a wonderful day.
-Kimberly