Only people who had a direct relation to Gaster (like Sans) can remember Gaster. If someone tries to speak about Gaster near someone that isn't allowed to remember him, they won't hear whatever is said.
Since a similar effect happens with Papyrus talking about the River Person while Undyne is around, it's only logical that the River Person and Gaster share some connection.