I did find one part of the flag code that says what you're saying - that the flag itself isn't to be used as apparel or draping.
However, it goes on:
"The flag should never be used for advertising purposes in any manner whatsoever. It should not be embroidered on such articles as cushions or handkerchiefs and the like, printed or otherwise impressed on paper napkins or boxes or anything that is designed for temporary use and discard. Advertising signs should not be fastened to a staff or halyard from which the flag is flown."
And also:
"No part of the flag should ever be used as a costume or athletic uniform. However, a flag patch may be affixed to the uniform of military personnel, firemen, policemen, and members of patriotic organizations. The flag represents a living country and is itself considered a living thing. Therefore, the lapel flag pin being a replica, should be worn on the left lapel near the heart." - i.e. suggesting that flag patches are something we explicitly want to reserve as something for our uniformed servicemen alone.