Yeah that's probably that...
I know someone who has this fear too, or more a need to be recognized for their work, which leaded to that.
That can go 2 way : either you don't care anymore about socializing, or you find a stable friend circle.
And if it can reassure you, I'm also an adult with no job and my little bro is probably gonna have one before me :(
And jeez, if you kicked out of your friend circle many times, I can understand your paranoia.
But I mean, you just need one good friend, and you'll feel way better :D
(Warning : do not rush for a friend, it's like the best way to get betrayed.)