It's explained in the prequel, Ballad of Songbirds and Snakes.
They make it clear that the Capitol directly controls whether or not a district thrives or starves through the supply chain, while living in luxury. The winner of the games actually wins more food for their District.