Just a guess, but it's probably because the normal road is grey, while the rainbow one is, well, rainbow. it's unfamiliar to them, which is why they decide to avoid it.
TL:DR. Rainbows aren't usually on roads, which makes the horse scared that it may be harmfull.
