Although this site is dedicated to virtual reality, you cannot discuss it without mentioning its very close cousin augmented reality, but what is it?
Whereas virtual reality immerses your senses completely in a world that only exists in the digital realm, augmented reality takes the real world of the present and projects digital imagery and sound into it. Augmented and Virtual Reality both fall on the continuum of mediated reality. Which is where a computer system modifies our perception of reality versus the “real” world.
As you can probably deduce this means many things qualify as augmented reality. The heads up displays we see in some aircraft and cars that may show you things like “distance to a target”, GPS position or your current speed are a form of augmented reality. Events with digital avatars of deceased musicians such as Michael Jackson and Tupac Shakur projected onto a screen using the Pepper’s Ghost illusion would also qualify under a broad definition of augmented reality.
However, when we hear about augmented reality these days it usually refers to a much more sophisticated, interactive and spatially aware implementation of the concept. Where digital objects such as 3D models or video are projected onto our view of reality as if they were really there.
How Does Augmented Reality Work?
The type of augmented reality you are most likely to encounter uses a range of sensors (including a camera), computer components and a display device to create the illusion of virtual objects in the real world.
Thanks to the popularity of smartphones, which have all the necessary components, they have been the place most commercial augmented reality applications that have been released.
In general the device looks for a particular target. This can be anything, but usually it’s a 2D image printed on something like a movie poster. Once the augmented reality application recognizes the target via the camera it processes the image and augments it in some way with pictures and sound. For example, you may see the movie poster spring to life and play a trailer for the film. As long as you look at the poster through the “window” of the display you can see augmented reality instead of plain old vanilla reality.
By using smart algorithms and other sensors such as accelerometers and gyroscopes the device can keep the augmented elements aligned with the image of the real world.
Using a smartphone or tablet computer as a “magic window” into the augmented world is one way we can relay this digital info to our eyes, but there are many other ways to achieve this.
Digital imagery can be projected directly onto physical objects. This is known as projection mapping and can be used to quite striking effect. For example, the Dyadic Mano-a-Mano uses projectors and Microsoft Kinect sensors to provide the user with 3D digital imagery projected directly onto the environment. The user doesn’t need to wear equipment or use any devices. Interaction with this system is highly natural and intuitive.
Projection mapping for augmented reality
Projection mapping as an augmented reality method has a lot of potential, but it requires a controlled and mapped space in order to work. The method that is most likely to supplant smartphone augmented reality as a common implementation outside of the laboratory is one that uses head mounted systems. This is where virtual and augmented reality really begin to converge, as there is no real reasons why the head mounted systems used by both technologies cannot be cross-functional. Indeed, head mounted systems that use smartphones to work often have something known as a camera “pass-through”. In other words, although you can’t see anything other than the screen of the head mounted display (HMD) it can show you the outside world via the rear-facing camera of the phone. This of course allows for augmented reality without the need for a handheld device. However, unless specially designed against it, this method leaves one feeling a bit disconnected from the experience, since the camera’s perspective and lack of depth perception don’t quite gel compared to what the naked eye sees.
One way to get around this is by using a system as found in the Google Glass and Microsoft Hololens. Both of these devices use something known as a “prism projector”. The eyes of the user look out at the world unimpeded, but digital imagery is projected into the prism projection system that sits between the eye and the outside world, making it appear as if those objects are really there, sitting on a table or hanging against the wall. The way this HMD achieves this is complex and fascinating and it discussed more completely in our Hololens article.
There are many ways to achieve the goal of augmented reality, but as you can see the end result is that we see digital information blend with the analogue world. Something that has many, many applications. Some of which we’ll take a closer look at.
Applications of Augmented Reality
Augmented reality has a wide range of applications in several industries and thanks to the rise of consumer smart devices and overall computing technology developments it now has lots of potential in the mainstream consumer space as well.
The two areas where we have seen a lot of commercial development in augmented reality are education and gaming.
The two biggest mainstream video game consoles, the Xbox and Playstation, have included augmented reality capabilities for the last two console generations. These game in the form of the Kinect (for the Xbox) and Playstation Eye or Camera (for the Playstation 3 and 4 respectively). Because you’re facing both the camera and the screen in these implementations are more like augmented reality mirrors, where you see yourself “in” the game and can interact with game characters that look to be in the same room as you.
Mobile augmented reality games are also not rare, and can be found on smartphones, tablets and handheld consoles such as the Nintendo 3DS and Playstation Vita.
Seeing the potential for augmented reality in education isn’t hard. It’s being implemented in fields such as medicine where students can benefit from live 3D models. It’s possible to use existing learning material (such as pages from a textbook) as targets for augmented reality. So when viewed through the lens of a smartphone you can see that picture of an engine animate in an engineering textbook or a working 3D model of a beating heart that you can walk around of rotate by hand.
In medical practice augmented reality can project information directly onto the body of a patient. For example, the Veinviewer system projects a real-time image of infrared vein scans directly onto the patient’s skin. Creating the impression that the skin is transparent. This allows the clinician to “see” the veins directly.
Military use cases are also quite clear, since soldiers wearing heads up displays (HUDs) can see information tagged onto objects in the real world. Radar information, orders or any other relevant sensor data from devices on the network that can provide it. Enemy and friendly positions are of course also useful to know. Augmented reality clearly has a bright future in military applications.
Mobile phones especially the iPhone use augmented reality apps which allow you to view computer generated images that have been superimposed over real world images. An example of this is an app which helps you to find a restaurant: it does this by displaying restaurant signs/logos as you move in a particular direction.
Another useful type of app is a golf GPRS system which helps golfers around a course. It displays yardages for each of the 18 holes, shows where the hazards are, e.g. bunkers and advice and support on improving your game. If you are golfer then this app will appeal to you immensely – look for the Golfscape Augmented Reality Rangefinder from the Apple store.
Augmented reality is also used in marketing and advertising as a means of enhancing certain aspects of a product in order to make it more attractive which will boost sales. This is discussed in more detail in our augmented reality marketing article.
10 apps that enhance reality
1. Prynt
This crowdfunded adaptor attaches to your smartphone, allowing you to instantly print out 2x3in photos. Also when you take a picture, the Prynt app automatically records a digital video and creates a link to that photo on your phone.
2. Bamzooki
This CBBC programme, broadcast between 2004 and 2010, provided an early large-scale glimpse into the potential of AR. Children could create digital creatures on a website that were then pitted against each other in a TV studio, while the audience watched via a graphics system that mixed the creatures into the real world. There is talk at the BBC of a similar programme returning, one that potentially allows children to pit their creatures against each other on the kitchen table or in the playground.
3. Lego
Most Lego stores now feature widescreen TV on the wall, relaying a continual live feed of the view directly in front of the screen. If you step into the camera’s field with a box of Lego, a virtual, animated representation of the kit inside the box will come to life in front you.
4. Inkhunter
This simple app lets you see what a tattoo will look like on your body by superimposing the image on to the camera feed from your smartphone. The app comes with a range of tattoo designs that can be edited and shared with friends. It’s also possible to upload your own designs to see how well they suit you.
5. Star Chart
An elegant and hugely popular AR app for smartphones that annotates the night sky. Simply angle your phone toward the heavens and, by using GPS positioning, the app will label the stars, moons and planets above you. A quicker and potentially more effective way of learning about our galaxy than any text book.
6. Snapchat
In the past year, the social media platform, whose videos are reportedly watched more than 6bn times a day, subtly shifted its focus on to AR through a selection of lenses. These lenses modify the phone’s camera feed in humorous ways, swapping people’s faces, “embiggening” their eyes, allowing them, for example, to puke rainbows. Snapchat became the first company to make a profit from AR on a large scale by selling and allowing companies to sponsor lenses.
7. Google Translate
Google’s revelatory translation technology, underpinned by computer learning to constantly improve its interpretation, can now be used to annotate foreign text on the fly, using your phone’s camera feed. While the technology isn’t great at handling reams of text, particularly in non-roman scripts, it can prove invaluable at handling signage.
8. Arnatomy
Perhaps the closest to Tom Caudell’s original vision [see main story] for AR, ARnatomy is able to identify and digitally label bones and muscles for medical students.
It uses the camera to identify replicas of human bones, and when the user places a bone in front of the camera, it identifies which bone it is and adds visuals pointing to the parts of the bone, such as where muscles attach.
9. Smartspecs
SmartSpecs enhance the visual appearance of everyday objects to augment vision for partially sighted people. By a combination of a 3D camera and Android-powered software, the specs highlight the edges and features of nearby objects – from walls, tables, doorways, signposts, buggies, and even faces – to enhance visibility.
10. Quiver
This relatively simple yet effective use of AR brings colouring books to life. Print out the black and white drawings, colour them in in the traditional manner. When you view the completed picture through your phone’s camera using the Quiver app the picture comes to animated life on screen. Touch the screen and you can interact and play games with the character you have created.
Conclusion
Augmented reality is likely to worm its way into our daily lives more and more in the 21st century. Once wearable computers become more common it won’t be strange to see people interacting with and reacting to things that aren’t there from your perspective. Thanks to technologies such as augmented reality the way we work with computing devices and think about the divide between digital and analogue reality is likely to change fundamentally. Nothing is stopping you from experiencing augmented reality for yourself today though. Just hop onto your smartphone’s app store and search for “AR” apps. There are plenty to try, many of them free.
