Forget about “virtual reality” for the time being; the new kid on the block is “augmented reality”. If the imaginative boffins in Japan have their way, we will soon no longer have to search out information about our immediate environment; it will just pop up in our field of vision when necessary. And like many, this technology has already been anticipated by videogames.

AR, as it’s known for short, aims to combine research in wearable computing with high-speed rendering engines in order to “augment” the reality that we ordinarily experience with context-targeted, computer-generated data. The applications are widespread. A brain surgeon, for example, might be able to “see” his patient’s head made transparent, together with a three-dimensional model of the brain to be operated upon, rendered from a previous scan and overlaid on the visual scene with precise calibration. A fighter pilot flying low in heavy cloud could nevertheless “see” realtime, accurately modelled terrain through his cockpit window. On a more trivial level, you might be walking down the street and pass a cinema, whereupon a list of film times and seat availabilities would immediately wink into existence before your eyes. In effect, AR could function as an instantly accessible, global and up-to-the-minute encyclopaedia wherever in the world you travelled.

Augmented reality is a spectacular kind of cyborg dream: a utopia where our very senses - our only way of directly interacting with the world - are themselves enhanced by information engineering. Some versions of it already exist. Military aircraft have already had a specialized kind of augmented reality for decades, in the form of the pilot’s HUD, or head-up display, whereby cockpit data is projected onto the windshield (through an old mirror trick known as Pepper’s Ghost) so that he doesn’t have to look down at panels of dials and gauges all the time. (This has long been paralleled both in serious flight simulations and in more arcade-style games, as with the tumbling red squares of Wip3out’s missile-lock system.) And sci-fi writers thought of it a long time ago, too: remember the Bionic Man and his special zooming eye. AR isn’t a brand-new idea; it’s just that only now is technology making it a feasible mass-market application in the next decade or two.

And as it develops it would do well to notice the creative use of AR in videogames, which have invented ever more useful ways of presenting information as a coherent part of the gameworld representation, rather than relying on marginal instrument readouts. A simple example of the way this can actually be justified in the game’s fictional context is in Deus Ex, where J C Denton’s nanobiology enables him to distinguish friend or foe from a distance when the player passes crosshairs over the target.

Crosshairs themselves, in fact, are a type of in-game augmented reality too. After all, it is clear from the perspective in which weapons are drawn that Denton, Dark or your chosen Quake 3 avatar is not sighting carefully down the barrel all the time. So the crosshairs are in one sense an arbitrary, artificial aid to the player’s aiming. Yet we can also think about them in terms of augmented reality, and surmise that future real-world guns with onboard computers that communicate with the subject’s optic nerves will be able to recreate precisely this situation. In this way, many things that currently look “unrealistic”, or like clunky fixes, in videogames might just turn out to have been ahead of their time. When real-world technology catches up with them, they’ll be seen to be perfectly believable.

A nice aesthetic paradox is that videogames in fact gave us the augmentations long before they could ever give us the reality. Take the brutalist, glowing green-and-red wireframe of Battlezone. At the time, it was in effect the best approximation to reality that videogame technology was able to deliver. But it managed to make a virtue out of its abstraction, persuading the player that he was a soldier in some futurist war in which this computer-generated image intensified and made navigable some dark, messy, hostile environment that was really out there. Now that wheel has come full circle: real military research into AR is providing infantry soldiers with Battlezone-style revelations of hidden enemy units on their helmet-mounted visor displays, overlaid onto the subject’s direct view of the environment. And games themselves can now choose to use wireframe as AR: G-Police 2, for example, cannily attempted to justify its horrendous draw distance by pretending that the environment really was extremely foggy, and that luckily your gunship was equipped with AR instruments that drew far-away buildings in ghostly outline.

There is another set of visual paradigms in games that do not strictly qualify as true AR, yet they might provide models for future augmentations that can be overlaid onto a direct view - the delicious apparatus of night-vision, remote cams, sniper sights and so on. Brilliantly manipulated by Metal Gear Solid and then taken to wonderful extremes by Perfect Dark, with its psychedelic wall-defying Farsight and weirdly claustrophobic infra-red goggles, these quasi-augmentations enhance even further the organic illusion of the gameworld’s actual existence. For if we can observe the same environment in many different ways - thermally, through X-rays, zooming in and out at will - we naturally get a sense of the world being more real, because it opens up to such manifold visual probings, than if we are constantly bumping up against magnified textures and operating in a world that confesses to only one informational scale.

It will be fascinating to compare the developments of videogame AR with its real-world counterpart. For if it is true that AR-style devices in games tend to increase the illusion of a reality that is already there to be augmented, it might work the opposite way in real life. Will we come to rely on the informative comfort of real-world AR so much that the naked, unaugmented real world comes to seem a shadow of its former self?

Leave a comment: