Biden called him xenophobic, yes. But I don't believe anyone called him racist for that particular travel ban specifically. They were addressing Trump's scapegoating of China for his own mistakes in the weeks following of the reported outbreak in his own inability or incompetency in preventing or lessening the spread of the virus in the US.
There was confusion by the admiration's policy that the Governors were to be directly responsible for getting the supplies without the federal government's help. When this was reported to the media, the administration scrambled to rectify that policy. It was less that the Governor's failed to utilize fema, and more the federal governments unwillingness to commit to aiding the individual states. Yes, the states could get these supplies on their own but they'd be competing with other states. And what didn't help was the Trump administration absolutely refusing to commit to fully supplying a state if it came up inadequate in it's own supplying. The Trump administration basically half-assed it. They elected for more a hybrid system. Providing inadequate quantiles of supplies to some states. Some got more than they needed, most got less.
In fact, there were even accusations of Trump favoring red states over blue states. And that may be partly true as the moron did say something to the effect of, "You have to treat us well." However, it also could be that most blue states are more densely populated than red states, and the Trump administration tried to seem like it wasn't playing favorites at all by giving each state the same amount of supplies equally without any regard to a state's population. An unfortunate, and deadly, miscalculation.