What exactly is augmented reality?

As we progress in the digital age, the world of "The Matrix" starts to feel more like reality than a science fiction tale. This is largely due to ongoing advancements in AR technology by engineers and developers. But, what is augmented reality?

Augmented reality is a type of mixed reality that merges digital elements—such as immersive visual effects, interactive haptic feedback, and sensory projections—into the physical world. If you played Pokemon Go, you’ve already seen augmented reality in action.

This once immensely popular mobile game allowed users to explore the world around them through their smartphone cameras, while overlaying game elements such as onscreen icons, scores, and the elusive Pokemon creatures. These digital items appeared as if they were part of the real neighborhood. The game was so captivating that it had millions of players—both kids and adults—wandering (and occasionally stumbling) through their own backyards in search of virtual treasures.

Google SkyMap is another popular AR application that overlays information about stars, planets, and constellations as you point your smartphone or tablet's camera towards the sky. Wikitude, another AR app, lets you point your device at a landmark or object to instantly retrieve information about it. If you’re looking to visualize how new furniture would fit into your living room, the IKEA Place app overlays images of new furniture, like couches, to help you decide before making a purchase [source: Marr]

Augmented reality isn’t just for smartphones. Its applications span various fields, including business, healthcare, and even the military.

The U.S. Army utilizes AR technology to develop advanced training simulations for soldiers. One such program is the Synthetic Training Environment (STE), which is designed to improve soldiers' decision-making capabilities on the battlefield. AR glasses and headsets may also enable military personnel to process information at unprecedented speeds. In the business world, AR is also proving to be beneficial—take the Gatwick passenger app, for example, which helps travelers navigate crowded airports with the help of AR.

The potential applications of augmented reality are vast and ever-expanding. The only unknown is how quickly developers will be able to integrate these features into the everyday devices we use.

Enhancing Our World

At its core, augmented reality involves overlaying graphics, sound, and other sensory enhancements onto our physical environment in real time. While this concept may sound straightforward, television networks have been using similar graphics for years. However, augmented reality goes beyond anything you've seen on television broadcasts. New technologies like RACEf/x and the super-imposed first down line in televised U.S. football games, created by Sportvision, come close but only display graphics from a single perspective. Future augmented-reality systems will be able to adjust graphics based on each individual viewer's viewpoint.

Some of the most groundbreaking augmented-reality work emerged from research labs at universities globally. At the February 2009 TED conference, Pattie Maes and Pranav Mistry introduced an innovative augmented-reality system they developed as part of MIT Media Lab's Fluid Interfaces Group. Named SixthSense, this project, though currently stalled, provides a foundational understanding of the basic components used in many augmented reality systems.

These elements were combined into a wearable device resembling a lanyard that the user wore around their neck. Additionally, the user wore colored caps on their fingers, which allowed them to interact with the images projected by the device [source: TED 2009].

SixthSense was groundbreaking because it utilized simple, affordable off-the-shelf components costing around $350. Its projector could turn any surface into an interactive screen. The device worked by using the camera and mirror to scan the surrounding environment, sending the image to the smartphone, which processed it, collected GPS data, and retrieved online information. The projector then displayed this data on surfaces like a wrist, a wall, or even a person. Since the camera was worn on the user’s chest, SixthSense enhanced whatever the user focused on. For instance, if the user picked up a can of soup in a store, SixthSense would overlay information about the product, including ingredients, price, nutritional facts, and customer reviews.

By using his finger caps—Pattie Maes mentioned that even fingers painted with different nail polish colors would work—a user could interact with the projected content, which was then captured by the camera and processed by the smartphone. If the user wanted to see more information about the can of soup than what was shown, they could manipulate the projected image to, for example, view competing brands. Unfortunately, the SixthSense project went on a long hiatus and is unlikely to be released commercially. However, many other AR technologies are stepping into the spotlight [source: Vulcan Post].

Smartphone Augmented Reality

In the Netherlands, smartphone users can download the Layar app, which uses the phone's camera and GPS to gather data about the surrounding environment. Layar overlays information on the phone's screen about nearby restaurants, landmarks, and more. You can even point your phone at a building, and Layar will tell you if any businesses in that building are hiring, or it might display photos of the building from Flickr or share its history on Wikipedia [source: Ensha]

Layar isn't the only app of its kind. In October 2018, Mural Arts Philadelphia launched a massive, interactive outdoor mural. Viewers could point their smartphones at sections of the mural, where they would see various holograms and hear corresponding music, creating an immersive art experience. In a science fiction fantasy come to life, movie fans can now play "Star Wars" Holochess directly on their phones, complete with cutting-edge graphics and sound [sources: Dickinson, Matney].

Healthcare providers may soon turn to smartphone-powered AR. Tissue Analytics, a company specializing in medical tech, is refining an app that helps doctors and nurses identify specific types of wounds using their phones for quicker diagnoses and more effective treatment [source: Comstock].

An app called Augment allows users to project almost any new product into their physical surroundings. Whether you're considering a new recliner or lamp, you can "see" the product in your home and access all related details and reviews before committing to a purchase. AR Compass Map 3D takes mapping to the next level, combining compass and map overlays with your camera to create an immersive 3D map, guiding you wherever you need to go [source: NewGenApps].

Similarly, Total Immersion, a company specializing in AR, offers a wide array of applications for both business and entertainment. Want to know how new glasses frames will look on your face before buying them online? Use the app to virtually try on the frames and discover that horn-rimmed glasses might not be your style after all [source: Total Immersion].

Then there's the iconic Pokemon Go, a game that took the world by storm in 2016, allowing players to hunt for virtual creatures scattered across real-world locations through their smartphone or tablet.

Augmented Reality in the Military

The military was one of the first to embrace gaming technology, recognizing its potential for realistically training soldiers in combat situations, but in controlled, safe environments. The same is expected for AR technology in the future.

Soldiers will increasingly immerse themselves in highly realistic battlefield environments, enhanced by helmet-mounted displays, smart glasses, and more. The military AR market could be valued at $1.4 billion by 2018 [source: VisionGain].

Arcane Technologies, a Canadian firm, has provided augmented reality gear to the U.S. military. The company manufactures a head-mounted display, similar to the ones once envisioned to usher in virtual reality, which overlays essential information onto the real world. Imagine a group of soldiers in Afghanistan conducting a reconnaissance mission at an enemy hideout. With AR technology, their head-mounted displays could project satellite views, blueprints, or even drone footage directly into their line of sight.

In a curious twist of fiction turning into fact, many of these military uses are becoming possible, thanks in part to the advanced physics programming and calculations pioneered by the first-person-shooter gaming industry [source: Nichols].

Having explored some of the current and emerging applications of augmented reality, let's now delve into the technology's limitations and what the future might hold for it.

The Mobile AR Revolution

While there are countless examples of smartphone-based AR apps, the reality is that many of these programs rise and fall quickly in popularity. What's truly noteworthy is that manufacturers, confident in AR's future, are designing devices specifically to enhance the AR experience. ASUS, for example, introduced the Zenfone AR with the hopes of making a significant impact in the emerging AR-focused market [source: ASUS].

Tech giants Apple and Google continue to refine their mobile devices to meet the growing demands of AR-specific software. With increasingly powerful processors in iPhones, iPads, and the wide range of Android smartphones, these compact devices are now capable of running complex, data-heavy apps, including those powered by augmented reality. Coupled with the advent of faster 5G networks, these devices will soon be able to handle massive amounts of data, resulting in a more seamless and advanced AR experience, whether you're inside a building or driving down a rural highway [source: Tardiloli].

Social media platforms are gearing up to take full advantage of AR's rise. Facebook, for example, has developed its own AR Studio to support the creation of AR apps designed specifically for its ecosystem. Additionally, Facebook is working on its own AR glasses. Meanwhile, Google is promoting its Tango AR platform, which features visual search capabilities via Google Lens, along with a range of AR camera tools. Apple is also in the race, providing developers with ARKit to build AR apps for iPhones and other iOS devices [source: Kahney].

The on-again, off-again saga of the Google Glass project reflects the unpredictable nature of augmented reality today. Google first introduced its AR-enabled Glasses in 2013, offering users an augmented reality heads-up display of their surroundings. However, the project stalled in 2015. But in 2017, the company revived the glasses for business applications. By 2018, a company named Brain Power began selling the Google Glass as part of a program designed to aid people with autism, helping them improve social interactions and encouraging positive engagement with their environment [source: Kronk].

Limitations and the Future of Augmented Reality

Augmented reality still faces a few hurdles. One major challenge is that many people are reluctant to rely on their smartphones, which have small screens that aren't ideal for overlaying information. To address this, wearable devices like AR contact lenses and glasses will offer users a much broader and more convenient field of view. With these devices, the limitation of screen size will vanish. In the near future, you could play a real-time strategy game on your computer, or invite a friend over to play AR games on the tabletop, all thanks to wearable AR glasses.

There's also the risk of being overwhelmed by information. Just as smartphone and internet addiction are concerns, relying too heavily on augmented reality could cause people to miss out on real-world experiences. For instance, some might prefer using AR apps on their iPhones instead of a tour guide, even though a live guide can offer a richer, more personal experience. Similarly, while a virtual plaque on a building may be convenient, it would only be accessible to those with certain technologies, while a physical plaque can be seen by anyone.

Despite these challenges, the possibilities are vast: With an AR-enabled phone, you could discover hidden facts about the city you’ve lived in for years just by pointing it at a nearby building or park. In construction, workers can mark locations for beams or structural inspections using virtual markers, saving on materials. Paleontologists could leave virtual notes on bones while assembling a dinosaur skeleton, artists could create virtual graffiti, and doctors could project digital X-rays onto mannequins for a more immersive experience.

We’ll continue to see a blend of AR applications. For example, companies will release simple tools like the AR toothbrushing game from Dixie Cups, designed to teach kids proper brushing techniques. At the same time, businesses and research facilities will find new ways to boost productivity, especially as the population ages. All these innovations will help push the boundaries of AR and make it more widely accepted as a valuable tool in many fields [sources: Odell, Abraham and Annunziata].

In the coming years, we are likely to witness significant advancements in augmented reality, with progress in software, hardware, and a wide range of new applications. The future of AR looks incredibly promising, so much so that you might just need both sunglasses and AR glasses to keep up with it.