While the world saw the device as a failed console, he saw it as an exciting toy.
The latest Iwata Asks column details Shigeru Miyamoto's thoughts and involvement on the Virtual Boy. He starts off by explaining that he had no primary responsibility for the project, and therefore was unable to guide its development. This was due to his work on the Nintendo 64, as both products were in development simultaneously. He introduced the idea of 3D goggles to Gunpei Yokoi, who was in charge of the Virtual Boy following his development of the Game Boy.
Miyamoto saw the Virtual Boy as a novel toy, and in those terms the sales figures were very favourable. He thought that with about five really good games, the product could find its own market and evolve as a gaming platform on its own, rather than being seen as a gaming device primarily, and the "Game Boy successor" from the beginning.
He doesn't feel the product was a failure in itself, but that the marketing approach didn't satisfy the goals of the product and protrayed it incorrectly. Since he was working on the N64 at the time, he had no authority with the Virtual Boy project. The sales department sold it as a console like the Famicom system, and sales expectations were not up to the standards of typical Nintendo consoles.
To complicate matters, the N64 was also working on the display of 3D graphics (and the wire frame representation used on Virtual Boy was even considered for the N64), though Miyamoto did not find the images terribly appealing. He explained the reasoning for the types of graphics on each product:
"If nothing but wire-frame fighter craft had appeared and Mario and other beloved characters had never shown up, that would be a little sad. But if you only changed the depth of a 2D image of Mario, it wouldn’t bring out the real appeal of the Virtual Boy. So the Virtual Boy system was a complicated affair."