I think that Eggy and Xander could go toe-to-toe, but I think that even with these limitations, Eggy comes out on top.
Like I said, his intelligence can be exploited by a more intelligent person.
However, Eggy has more fighting experience, and ranged attacks like his trident and magic lightning and lasers, and even metal bending abilities, which I think gives him the edge in this confrontation.
Xander is only armed with melee weapons, therefore, with Eggy's speed, he could easily stay out of Xander's range and hit him with a barrage of ranged attacks.
Caption this Meme 