Well, originally the rainbow pride flag was just that - a rainbow. It was meant to represent just a multitude of people who are as a whole pretty colorful. Also, we'd have to update it constantly to include new genders or orientations and it would lose all of it's recognizability and therefore its uniting power.
The individual pride flags for specific genders and orientations usually do assign the colors some specific meaning though. But some colors can have different meanings in different flags so it's not like there is a certain color for every detail and you just put together the relevant colors to your sexuality and *poof* new pride flag. It's basically like real vexillology... 😅