Augmented, virtual and mixed: the new realities

While VR has arguably underperformed in the market to date, hamstrung by high headset costs and consumer apathy, AR has been the stealth technology that has quietly crept out and opened the gate.

It’s worth just pausing to check some definitions here, as there is the potential for some confusion. VR entails the projection of a wholly-generated 3D world into an immersive headset with every element created by computer graphics.

The same VR headsets, from the top range Oculus Rift and HTC Vive down to the Google Cardboard and smartphone combinations, can also be used to project 360-degree and 180-degree video.

AR is essentially a graphical overlay of CGI elements onto real world elements, typically generated by a video feed.

Pokémon Go is perhaps the most obvious example, and one whose $600m worth of sales from an incredible initial three-month burst last summer is one that has concentrated minds worldwide.

That is more money than the whole rest of the VR games market managed to earn over the course of the entire year, and almost singlehandedly led analysts Digi-Capital to predict that AR would have the majority of sales in a $108bn global ‘X Reality’ market by 2021.

There is also mixed reality to consider.

Sometimes referred to as hybrid reality, this blends together all elements into one, blurring distinctions between real world and visual objects so that it becomes meaningless, and also folds in advances in sensor technologies, optics and computing power to provide, in theory at least, a more realistic experience.

The devices driving AR

Of all the forthcoming devices that are looking to ride the AR wave, there are three in particular that showcase very different styles of development.

Microsoft’s HoloLens is probably the most well-known and, to date, available.

Following on from the glasses-based form factor pioneered by the ill-fated Google Glass, it’s a visor-based headset that uses two HD ‘light engines’ to project AR components onto see-through holographic lenses.

Four ‘environment understanding’ cameras, one depth camera, four microphones and an ambient light sensor mean that the generated images should match the local environment.

It’s been available to developers since March 2016 at a $3,000 price tag, but there is still no word yet on a genuine retail version, with speculation that this may even be postponed now to a third generation unit (the second gen will be skipped) with a rumoured 2019 release date.

Partly this is due to a lack of robust competition in the market as everyone else keeps to the R&D labs.

The second device is Magic Leap, the ultra-secretive Florida startup that managed to raise $793.5 million in startup capital in 2016 alone, accounting for $3 out of every $10 invested in VR/AR that year.

All in all, it has swallowed somewhere north of $1.4bn by now but, apart from a succession of genuinely stunning YouTube videos reportedly shot through a device’s lenses, very little is known about it and how its technology works.

To try and build a coherent picture of the current state of Magic Leap out of a myriad of disparate sources and you end up with a glasses-type device smaller than the current HoloLens, but tethered to a backpack, which projects images straight onto your retina and will be released this coming holiday season for $1,000 or more.

The third device though is much more easy to talk about as it’s pretty much every iPhone made since September 2015.

Back in July 2016, Facebook CEO and Oculus Rift owner, Mark Zuckerburg, responded to a lighthearted question over whether he had played Pokémon Go by saying that “the phone is probably going to be the mainstream consumer platform [where] a lot of these AR features first become mainstream, rather than a glasses form factor that people will wear on their face.”

This is becoming common opinion.

Analysts Digi-Capital listed five challenges to be surmounted before a truly mass market AR device can emerge — hero device status, all-day battery life, mobile connectivity, fecund app ecosystem and telco cross-subsidisation — that pretty much described the modern smartphone.

Apple has been nosing around the subject for a while now, but rather than bringing out its own iGlass alternative (at least yet) it is first deploying its new AR technology on any iDevice kitted out with A9 or A10 processors: in other words new iPads and iPhones and an installed base that the company reckons numbers in the ‘hundreds of millions’.

AR is going to be an integral part of the forthcoming iOS 11 release, which should be timed to coincide with the launch of the iPhone 8 in September.

Apple launched its AR App Development Kit, the ARKit, at its Worldwide Developers Conference in June and third parties have been beavering away with it ever since.

For anyone used to the simple superimposing of random cartoon creatures on the world around them, it represents a whole new level of sophistication.

The software is capable of automatically analysing the scene in the phone’s screen to detect horizontal planes such as tables and chairs, so that objects placed on them stay on them, or objects that roll off them fall off them.

Indeed, a demonstration at the WWDC from developers Wingnut saw a whole tabletop war take place, with soldiers and building rubble falling dramatically off the edge of the table and onto the floor below.

The iPhone is not the first smartphone to feature AR-boosted hardware, that was the slightly unwieldy Lenovo Phab 2 Pro, which was the first public outing for Google’s Project Tango technology. But it is the first genuinely mass market device out there and, crucially, can already tap into a large and very active development community.

Demos so far have ranged from the simple but practical — a virtual tape measure that you never realised was going to be so useful till you tried it — to the visually impressive and Unreal Engine-driven gaming demo mentioned above.

What’s key is that there’s a lot going on, giving the industry more of a chance of stumbling upon a breakthrough hit and igniting the AR mass market in the space of a single app.

If you’re looking for a timeframe for the wide scale introduction of AR, then 2017 is pretty much it.

What is largely unknown is how the transition from flat phone screen to glasses-based headset will work, if indeed it will work at all.

Consumer pushback against VR headsets has been stronger than anticipated, suggesting a lot of people didn’t really pay much attention to the way stereo 3D faltered in the marketplace.

But smaller, lighter, self-contained devices that let people interact with virtual content in real world environments?

That is a far more interesting proposition. Whether they will work out in the real world we should know in a couple of years, but certainly by the time they are released we will all be much more accustomed to rather more blended notions of reality.