Could this be the real future of 3D gaming?
While perusing the always relevant and usually humorous ranting of Tycho Brahe at Penny Arcade a few days ago, I came across an interesting link to the Website of a company that sells a very high quality 3D display. Although it's very expensive and mostly for scientific and medical purposes at the moment, I couldn't help but wonder what implications this could have on video games, even if it wouldn't be available at consumer costs for years.
You can examine the Website to get a good idea of exactly what the display is capable of and how it works, but I'll briefly summarize this here. The problem with displaying graphics in 3D is that there is no suitable surface to project the graphics onto. In 2D displays, beams of light are generally arranged so that they project onto a flat surface (or from a flat surface) allowing them to be viewed as a painting would be. The equivalent 3D display would display an image that comes to life in full 3D like a sculpture rather than a painting. The problem with a 3D display is that you can't have a 3D screen (say a cube) to project onto because only the outer faces would be fully visible, even if it was transparent somehow. Actuality Systems solves this problem by using a quickly rotating 2D surface in a glass ball for projection. The screen makes about 12 rotations per second (or 700 rpm) giving the projector ample time to display voxels (the 3D equivalent of a pixel) on the screen. The voxel must be flashed on the screen during an exact moment of rotation to have the light end up at precisely the right point in 3D space. The end result is a 3D image that can be viewed in full 3D from any vantage point (just as you might view a statue or sculpture), and that is crisp and bright like a standard monitor (as opposed to older holographic displays). Just to be clear, this isn't the first 3D display that's been designed and it may not be the best to date either, but I figured an explanation of it was in order since it triggered this article.
Now, Imagine that years (possibly decades) in the future, 3D displays are finally available at reasonable costs. Surely a good use of the displays would be for 3D video games right? Well, maybe not. After some thought, it seems to me that a 3D display would be a generally poor medium for videogames because of its inherent limitations. We take many qualities of traditional 2D displays for granted. Even though the picture is limited by the horizontal and vertical dimensions of the display, the depth of the display is another matter entirely. Because a 2D display has no depth, the illusion of depth must be created. This is a very natural process for photography and film because our eyes capture light in the same way on the surface of our retina. In computer graphics (and video games) elaborate algorithms have been developed to generate 3D images on flat screens. Given enough time and computer resources, an image can be calculated that resembles an image we might capture with our naked eye (although framed by a monitor). The key to this technique is projection. Images are calculated as projections onto a virtual surface which represents the light-capturing device in a camera. Projection allows virtually infinite depth to be displayed on a flat surface just as you can capture light for "as far as the eye can see".
Contrast this with a 3D display. You would think that a 3D display would be the perfect medium for 3D games, but in actuality, it's more suitable for 2D gameplay as you'll soon see. With a 3D display, you go from limited horizontal and vertical viewing with infinite depth, to limited horizontal and vertical viewing with limited depth. In other words, for a 3D display, when you look past the end of the display, you can't see what's beyond it (obviously). It's like a wall of enforced fog or blackness that can't be removed no matter how good a developer is. What's worse than this is the problem of camera movement. With a 3D display, you are the camera. The problem with this is that almost every 3D game ever made absolutely requires that you don't always have control over the camera. Imagine playing a first person shooter on a 3D display. In an FPS, the camera represents the player so it has to follow strict laws of physics (for example, it will generally be close to the ground, and it can only move as fast as your player is allowed to move). To enforce this on a 3D display, the game would have to instruct you to sit directly in front of the display and not move. This would give you a TV like view of the game with real 3D graphics instead of projected 3D graphics. It would work, but you would see nothing beyond the end of the display. Of course, there is the possibility that the last part of the display could be used like a 2D display. This way the game would show foreground objects in actual 3D and at the last possible point, the graphics would switch to projected 3D onto the flat rear "surface" of the 3D display. However, this isn't enough. The left and right sides, as well as the top and bottom of the display would have to be pasted with a similar 3D image (using today's methods). The image would have to be generated with knowledge of exactly where you are for it to look correct. If all is well, then when you looked into the front of the display, you would see an image that (surprise) looks basically like it would look on a normal TV. Anyone else who viewed the display would see a mess of color and light unless they viewed it from exactly where the player was sitting. As you can see, trying to simulate projection is pointless; it works much better on good old 2D displays. Why reinvent the wheel? This leaves you with what you started with: a 3D display that can create a sculpture like image with clear boundaries on all sides.
Not all hope is lost though. There's always the potential for making 2D style games with 3D elements. Nintendo's failed Virtual Boy is an excellent primitive example of this. While the VB display was far from a true volumetric 3D display, it had two or three 2D layers that gave it similar properties (although you're forced to view the display from one angle because of the goggles and orientation of the screens). That said, a volumetric 3D display would be great for Virtual Boy emulation! *chuckle* Aside from that, you have the potential for volumetric 3D puzzle games, games with a mostly overhead camera reminiscent of Pikmin, and 2D games with 3D graphics (perhaps a few others).
The big problem is that none of the games I mention require a 3D display. Every one of them could be done today with a regular TV. It could make them a little more interesting of course, but when the console (or arcade system) got nothing but puzzle games and 2D gameplay, it would become apparent that the display is more of a step backwards. A step backwards is not a good way to push new technology into peoples' homes. No doubt, we will see 3D displays used for games at some point. As I mentioned above, it would appear in arcades before anywhere else. In fact, it's already happened to be perfectly honest. Years ago, using primitive holographic technology, some games were made that suffered from the exact problems I mentioned above. The game I remember featured a western theme. It was a Sega game called Time Traveller which is now being rereleased (with 3D glasses!) for disc based systems. On the first level, you controlled a gunslinger and somehow had to take out a bad guy. I didn't beat the first enemy. Scenery included a cactus or two and maybe a tombstone. The gunslinger, the bandit and the cacti were just floating in space. There was no other detail. Gameplay was very limited because the characters were filmed with cameras so you had very little control. You basically just decided when to fire a shot (the firing was delayed too). It was barely a game. With a more capable display such as the one I've been talking about, some more interesting games could certainly be created, but I believe that they would still be mostly gimmicks. Maybe someone will eventually come up with a revolutionary game that can only be played on a 3D display, but it seems doubtful. Almost any type of game that you would think requires a 3D display could be simulated with one or more traditional 2D displays. Next time you play a 3D game on your TV or watch a movie, stop and appreciate the magic of projection.
If you'd like to discuss this editorial, please contribute to this thread in our (free!) forums. Although I'll be monitoring the thread, you can send email directly to me at David@PlanetGameCube.com if you prefer.
Finally, props to Penny Arcade and Actuality Systems for inspiring the article, and olubode (from the IGN Boards) for helping me remember the name of that ancient Sega game.