Well not every korok seed, for me I'm currently focusing on getting all the lightroots even though I have the main quest "Destroy Ganondorf" right there every time I press -, and in BoTW I focused more on getting every single armour set there was before I went to the castle...
The thing about koroks is that there're more than you'd ever need, so you don't have to hunt them down; you can just find them along the way...
I headcanon that the Legend of Zelda is a multiverse (MCU-esque) with every player's savefile as an seperate universe... So there were plenty of instances where Link just mucked around doing useful-but-not-important stuff and ignoring Ganon...