People would complain if it was just story elements.
Maybe if there was an item that wasn't mentioned until you found it. Something you don't really need, but is really cool to have if you take the time to actually go get it.
Like you go off on some random quest to fetch someone's dog that ran down some cave that everyone was too afraid to ever go down, and in the process of looking for the dog you find something that leads to a small dungeon hidden in the cave that leads to a treasure chest with a map written in some ancient forgotten language.
You have to take this map to some historian somewhere in Hyrule who would translate the map and give you the story of some ancient mythical sword/shield/armor that had magical powers that activated when it was combined with the Hyrulian Diamond (a stone hidden in a castle somewhere in the last 1/4 of the game). The sword/shield/armor was dismantled/broken and hidden across the lands but the exact locations were a secret. A secret held by the Hyrulian Historian Society, a secret society so secret, everyone assumes it's just an urban legend that has been passed down from generation to generation. Now you have to track down these HHS members, who are scattered all over the world and only meet in the Hyrule castle towns hidden underground temple on nights of the Full moon. You might see these members scurrying around town late at night(in dark robes) only to disappear down some dark dead-end alley, and you might even catch up to them and try to talk to them, but they will say nothing in return if you don't know the secret HHS phrase.
But of course you would never even hear about this side quest if you didn't wander further than necessary into a cave to rescue some dog for a little girl, and the side quest takes you away from the main course of the game and extends the length of the game by several hours by opening up hidden dungeons that you wouldn't have access to. Now instead of all those useless rupees in the random treasure chest, occasionally you would find a weird metal fragment with no real description. If you didn't know of the quest, you wouldn't know what these random fragments were or what potential they might have if you collected them all.