for me it's because i generally just hate milk itself... strangely not milk-based products though, i absolutely LOVE eating most kinds of yoghurts and i also drink hot cocoa (which ofc uses milk as a base) during dinner
Which would mean having to harvest the materials required to make it first, and in the case of cereal it would have to be grown first. It would seem there's always one more preceeding step.