Drop the "New" part and find Hampshire is a county in England. Same with most places early settlers were from that has "New" as a prefix. Even New Zealand is named after Zeeland, a Dutch province.
There's no "old" because they were just the original. You only name it "new" if there was an "old" though.
Wait until you look in England for "Essex" and "Sussex" but no "Wessex" (there used to be) or "Nussex". With Northampton and Southampton they are north and south but Easthampton is actually to the west of them and there's no Westhampton". Hampton is an area in London, to the east of these places. They're also hundreds of miles away from eachother and dont reference n,s,e,w areas of the same place. Confusing, isn't it?
