I'm not so sure if flying a broom is not tiring at all. I'm used to travel long distances on a scooter and yeah, I can be exhausted when I arrive. Since I have distances on my name where I had to travel 4 and even 6 hours, I really needed my breaks underway in order to be able to keep my mind on the road. I really found out that some laws that after 2 hours of driving you require 15 minutes of rest is there for a reason. And depending on how the wind is set, you can be even more tired. Driving against the win on a scooter makes your body instinctively use all your muscles and you have to or you'll blow off your scooter. This happens without you noticing, but takes a lot of energy, so all in all... I don't think broomflying is not tiring.
Of course remember Wood's words in the movies, just before Harry entered the pitch for his very first match: "I don't remember. I took a bludger to the head. Woke up in the hospital a week later"... ?
