Just finished listening to the episode. Great discussion.
I never owned an N64, so I didn't play Ocarina of Time until I got it as a pre-order bonus for Wind Waker. One thing that surprised me was how "creepy" the game looked at times, which I attribute mostly to the N64 graphics. The Biggorn's Sword sidequest is still my favorite sidequest of all time. It was quite rewarding.
I actually stopped playing for a year when I got to the infamous Water Temple. I used a guide on Gamefaqs because of how big of a pain it is. I got all the way to the boss's door when I realized I had missed the boss key. I didn't want to try searching for it, so I just stopped. When I did pick it up again, it was actually pretty easy to find.
To me, the most memorable part of Ocarina of Time is the final boss. It was epic and one of the most enjoyable I've ever played.