In establishments, I agree it is reasonable to expect masks. In California, they have been declared mandatory, everywhere.
Also, I know I breath more easily through single use surgical masks than the reusable cloth ones. If someone only had access to cloth masks, I could see their objection.
Also, don't overreact if someone pulls it down momentarily. The could be trying to get a deep breath to stave of an asthma attack, or a panic attack. (Even though you can technically breathe through a mask, the sensation of obstructed breathing can trigger various symptoms in those with anxiety).