The main problem with the N64 cartridges was their size, definitely. Not size as in memory capacity, but their shape.
These cubes could be integrated into a form of flash card like they have with Memory Sticks and SD memory, so the size problem can be forgotten.
Optical media's main flaw is the accessability of the media - mainly the delays that occur when changing between sections of disc. Sadly it's quite obvious when dual-layer DVDs change layers during a movie.
As for people saying that optical media would remain forever, they are stupid.
CDs superseded casettes which superseded vinyl, and DVD superseded video casettes. Something will replace optical media, its just that people are too ignorant to look into the past and notice.
I can see it influencing the gaming industry, but while Sony and MS push DVD as their media of choice for their consoles, I don't expect it to be widespread in the future. The RIAA may be interested in this though...