Note to eggy: This mean you can't make a better swordsman than the solar knight (reasoning of choice in comments)

Blobler doesn’t have a sword so he doesn’t have a reason to be in there

Solar knight/Xander has used the Muramasa for a century, even when the Calamity ended, some of his knowledge of the sword was forgotten but he still retained most of his knowledge, His Muramasa also contributes to the reason why he's the best swordsman, the Muramasa being light but deadly makes him a deadly threat and if the Muramasa doesn't work... he always has Gael's greatsword to annihilate his targets

The collector is the most powerful being, but being the most powerful being doesn't always mean you are the best in everything, for example swordsmanship. The Collector's sword (Sword of creation) is the strongest sword but that means that he doesn't have to rely on good swordsmanship to get the job done as the sword can also have projectiles or magic infused attacks, to sum it all up, the Collector is so strong he doesn't have to rely on good swordsmanship to be able to win with the sword

Yharim is one of the few beings who have defeated the solar knight in the past, Yharim's sword (True Tyrants Yharim's Ultrasword) is much stronger than the Tyrant's ultrablade or Tyrant's greataxe. Yharim's swordsmanship is also very powerful as he is able to defeat gods with ease, Yharim's swordsmanship falls short to Xander's and Collector's because of his depression and/or his rage.

the collector has a sword?

Yes
The swords of creation

Yes he does, the Sword of creation

"Something tells me he isn't gonna take too kindly to this...."