No, it's smart.
Housing costs have gone so high, that even states where people could afford a house or rented cheaply not too long ago have become unaffordable.
If these people can't get their acts together in their own countries, how is it that they're supposed to be able to do it here, and why should they? If they have any ability to to improve their own personal economic situation and perhaps others in the community as well, then wouldn't it be better served for them to do it back home for their fellow countryman, instead of abandoning their own blood for reasons of simple basic greed?
Also, categorizing these people as nothing but color coded lumps is just as racist, if not more so, than rejecting them for it. At least the ones rejecting them are being honest. Many, if not most of those people coming from those other countries are 'White,' and identify as that at home. So-called "brown people"? And how is identifying them soley as such not racism?