We cannot talk about augmented reality (AR) without first looking at the evolution of computers. Computers have an impact on almost every aspect of what we do. They dominate our financial world, manage the global transportation system and make the worldwide marketplace possible. They’re significantly expanding medical products and services delivery; they are changing education; they are fundamental development tools of the entertainment industry, and they are the driving force behind the Internet.
Initially, the link between the real world and the world of zeros and ones was a tenuous one. Text-based data input and output in operating systems like PC-DOS and MS-DOS were able to provide the requested information, but little else.
Graphical user interfaces on personal computers like Windows and the Mac OS were a significant improvement. They not only provided a richer user experience, they made it possible to represent real life objects in cyberspace.
The ability to create and manipulate individual graphical elements on a screen made it possible to build entire self-contained mini environments that evolved into today’s virtual reality. Graphical user interfaces didn’t make VR possible; they just facilitated its development. There were types of virtual reality before the advent of graphical user interfaces (they just weren’t called VR) and before computers for that matter.
The very first attempts to simulate some form of reality came in the late 1830s, shortly after the invention of photography itself, with the invention of stereo photography.
Virtual Reality Evolves
By the mid 1800s, stereoscopic slides were the rage. Refined ladies and gentlemen would sit in their parlors in their Sunday best viewing pictures of the Grand Canyon, the pyramids of Egypt and the Great Wall of China. Viewers would remark on how realistic the 3D pictures looked. “Almost like being there” and “As if I could step into the scene” were frequent comments. They were amazed. (The reality of stereoscopic slides was credited, in part, with starting the first worldwide tourist boom.)
Training systems utilizing simulated environments came along not much after that. Rudimentary flight simulators used for pilot training, for example, became available almost as soon as flight itself. By the 1920s, mechanically driven simulators were already in use. In 1934, the Army Air Forces bought its first six, adding another 10,000 to its inventory during WW II. They were responsible for saving countless airmen’s lives.
Then in the mid-1950s there were various attempts to create realistic entertainment experiences. Probably the best known of those was Morton Heilig’s Sensorama. In 1956, Heilig created a virtual environment for a film in which the viewer was given a simulated ride on a motorcycle.
The film was specifically designed to stimulate all the senses, to provide as realistic an experience as possible. It was intended to be seen by one viewer at a time, sitting on the motorcycle, with fan-produced wind blowing the hair, the smell of motorcycle oil and gas wafting, and 3D projected scenery passing by. While the film incorporated some advanced features, with only one viewer at a time being able to see it, it wasn’t particularly financially successful, and the process never caught on.
Bringing VR to the Masses
While personal computers and their graphical user interfaces didn’t make VR possible, they did popularize it. They made it more sophisticated with complex 360º computer-generated scenes, highly stylized sophisticated controllable objects and a range of interactive control devices. These range from touch pads to sensored gloves to full body suits. All these contributed to heightening the sense of reality. And the proliferation of personal computing devices brought virtual reality within the budgets of the masses.
With VR, viewers were able to put themselves into that created reality, look around in any direction, manipulate the objects they saw and interact with anyone else that might have virtually been in that same self-contained environment. The ability to interact with others in that virtual environment not only made multi-player video games possible but also sophisticated simulation and training exercises.
Still, whether training in an expensive mainframe-driven commercial flight simulator or exploring computer-created worlds on an affordable PC, until relatively recently, all the content was still computer based. You weren’t able to take that computerized environment and place it into the real world.
Augmented Reality: A World Apart
With augmented reality (sometimes referred to as “mixed reality”) you can. You can take sophisticated, fully manipulable elements of computer environments and superimpose them onto the real world. For example, it’s possible to project the keypad of a smartphone onto your hand, and then touch-dial a phone number on the projected image to complete a call. Or you can drive through a city and have the names of each crossing street appear, and get relevant information about that street or neighborhood.
That’s the fundamental difference between VR and AR: the world that you’re operating in.
Just as with VR, which requires specialized equipment, AR also requires specialized devices. But unlike VR devices, they don’t shut the real world out. They, in fact, embrace it. AR devices combine both natural and synthetic light. Synthetic images are overlaid on see-through devices or the real world, and virtual data are merged on a device with a screen.
Unlike VR devices that are generally bulky and tethered to some desktop or handheld computer, AR devices are generally lightweight and support wireless Bluetooth and Wi-Fi. Google Glass is a good example. AR devices are expected to get even smaller and lighter. At some point, something as small as a contact lens will be able to serve as an AR viewing device.
Furthermore, AR requires massive computing power, so viewing devices are paired with processing power. These devices include a high-powered central processing unit, an advanced graphical processing unit and internal memory. They also feature a Bluetooth/Wi-Fi communication chip and a global positioning system chip. At times, they emply gyroscopes, accelerometers and magnetometers to measure things like orientation and velocity. They also require sensors, cameras and even optical projection systems that make interactivity between the virtual and real worlds possible.
Smartphones as AR Devices
Fortunately, there’s a category of relatively affordable devices on the market that contains many of these capabilities: the smartphone. For now, the smartphone is an ideal solution. Smartphones are mini supercomputers that have the processing power to make AR possible. And they contain many of the components—such as GPS, Wi-Fi and Bluetooth—required for AR. Smartphones and to a certain degree tablets will, for the near future at least, be the driving force behind AR.
One of the best examples of the power of smartphones in AR is the craze that introduced most kids (and their parents) to AR—Pokemon Go. With nothing more than a smartphone, kids could hunt down superimposed Pokemon as they explored the real world.
The potential for AR is endless. Imagine finding a set of instructions to assemble an entertainment system and seeing they’re in Japanese. AR can automatically translate them to English in real time. Or having everything you’ve ever wanted to know about the Protestant Reformation at your fingertips as you retrace the footsteps of Martin Luther through eastern Germany.
To handle those tasks and seamlessly integrate the data and virtual elements with the real world requires not only powerful hardware but also very sophisticated software. Software that’s driven by artificial intelligence (AI). Without AI and deep learning (the attempt to make computers respond somewhat more like biological nervous systems than machines), there wouldn’t be AR. AI makes the sophisticated image, pattern and speech recognition systems required for AR possible.
AI also impacts physical environment mapping so that real world and virtual world elements are displayed in real time, as well as sophisticated virtual character modeling and adaptation, so that AR responses are more life-like.
With AI, the objective is not only to display information or images but also to actually learn from the input and responses of the user for what’s expected of it. As a result, the created elements that are integrated into the user’s real world become more and more relevant.
Varying Forms of AR
Augmented reality is activated in different ways, including marker-based, marker-less and projection-based methods.
Marker-based AR utilizes a camera and something like one of those boxy quick response (QR 2D) codes to call up desired information. This is sometimes referred to as image recognition AR. Applicable for almost any data delivery, it’s particularly effective for getting information on products and out to potential customers.
Marker-less AR is GPS based. If a tourist is standing in front of the Eiffel Tower, the history and data of the landmark would be displayed on the viewing device.
As the Pokemon Go craze demonstrated, it’s possible to get more than information and text delivered in AR. It’s possible to superimpose VR elements onto the real world. That’s beneficial for commercial purposes. A furniture shopper wants to see how a specific couch looks in his living room. With AR, he can see how different couches look in that space before buying. (Superimposition AR is generally considered a separate category from other AR types.)
There’s also projection-based augmented reality. That’s the form of AR that makes it possible to dial the number on a keypad projected onto one’s hand. With projection AR, a camera reads how the projected light is altered by human interaction to determine what will be done.
Even though AR has been around for a few years, it’s still in its infancy. It will become increasingly important in fields from space exploration and scientific research to medical procedures and emergency responder training. Another application is game-development opportunities, as hardware components are refined and supporting software applications become more sophisticated.
In fact, Digi-Capital’s latest Augmented/Virtual Reality report projects that mobile AR will be the driving force behind a $108 billion VR/AR market by 2021, accounting for $83 billion of that market!