This was the year of VR, until AR stole it

This was supposed to be the year of virtual reality, but barely had 2016 started when Microsoft threw a spanner in the works with the announcement of HoloLens. Rather than taking us to a virtual world, Microsoft’s headset pulls virtual objects into our own. Microsoft calls these objects Holograms, much to the chagrin of hologram enthusiasts, but most people know them as tenets of mixed, or augmented, reality. It’s already being touted as the next next big thing.

Of course, 2016 was full of VR. With spring came the retail launch of the Oculus Rift and HTC’s Valve-endorsed Vive. Both require two things: a lot of cash and a lot of power. The Rift costs $599 while the Vive is $799 (including controllers and tracking accoutrements). But then you need to factor in the price of a PC that can support the high-fidelity, high-speed visuals VR requires. A typical all-in price started from $1,500, putting it out of the range of all but the most ardent of gamers. That price has dropped and will continue to drop as cheaper, better graphics cards are released.

There are no firm figures for how many VR kits have been sold. Steam statistics suggest that just 0.34 percent of its users in November had a headset. Even counting gamers who don’t use Steam, that would likely put the total figure sold across both Vive and Oculus at well under a million. That estimation is in line with VR analytics group SuperData Research, which projected around 450,000 HTC Vive sales and 355,000 Oculus Rift sales for 2016.

Just as Oculus and HTC should’ve been dominating the news cycle, Magic Leap, the secretive Google-backed mixed reality (MR) startup, finally broke cover with a Wired feature. Magic Leap is basically promising to do the same things as HoloLens, but better.

Details are scant, but rather than projecting images onto a portion of a giant helmet (like Microsoft’s headset), Magic Leap will beam light into your eyes, using a system called Dynamic Digitized Lightfield Signal to give these objects depth and solidity. The company has yet to show off any hardware or software or even suggest a year when its tech will be ready, but it’s nonetheless one of the best-funded startups around. Wired’s Magic Leap feature came in April, within weeks of the Vive and Rift launches. The timing was obvious, and the message was clear: There’s something better around the corner.

In the meantime, an ex-Google startup with a couple dozen employees was preparing to steal everyone’s attention with a mobile game. I’m talking about Niantic, of course, and Pokémon Go, which was undoubtedly the hit game of the summer, if not the year.

Somewhat erroneously referred to as an augmented reality (AR) game, Pokémon Go is better described as a location-based game, like geocaching, with a pervasive layer on top. Definitions aside, there can be no doubt that AR has been a big part of its huge success. When catching Pokémon, players are shown a live feed from their device’s camera with a monster overlaid. Hundreds of thousands of people shared these images on social media, helping spread intrigue about the game.

Before long, packs of Pokémon hunters were roaming New York, London, Paris and other locations around the world, searching for new monsters and using an AR system to help catch them. Unlike Niantic’s last game, Ingress, this wasn’t just geeks and gamers. I can count on one hand the number of Ingress players I know. With Pokémon Go, I can count on one hand the people I know who didn’t play it. My 64-year-old mom played. My 10-year-old son played. It felt like, at one point, almost everyone gave it a shot. By the time Niantic announced an Apple Watch app for Pokémon Go, the game had already been downloaded 500 million times. That’s a ridiculous number.

Of course, crazes rise and fall, and it’s safe to say that Pokémon Go is, if not gone, seemingly on its way out of the public’s imagination. But its impact remains. My colleague Kris Naudus referred to Pokémon Go as AR’s aha moment, and I agree. For a fleeting minute, the game brought a little Pokémon magic into our world. It’s one of the most basic implementations of AR around, but we found it compelling. That should be encouraging for Microsoft, Magic Leap and any other company that’s planning a mixed or augmented reality product.

So where does that leave virtual reality? Well, there are still plenty of headsets out there, and VR is not going away anytime soon. Sony launched the PlayStation VR just a month ago, and it’s expected to equal Vive and Rift sales combined by the year’s end. It’s not that PSVR offers a better experience than its PC-based cousins. It’s just a lot cheaper — $399 to $499, depending on your needs — and has a way bigger reach. Steam stats suggest little over 10 percent of PC gamers have a VR-ready computer. Every PlayStation 4 owner can plug in a PSVR and get started. That gives Sony somewhere between two and four times the potential audience.

And even PSVR’s prospective audience is dwarfed by the potential market for smartphone VR. Google has sold cheap Cardboard viewers for a couple of years, but this year the company announced Daydream, a new initiative to bring a more premium VR experience to mobile users. Daydream View is a $79, comfortable headset sold with a bundled motion controller. At present, only Google’s Pixel and the updated Moto Z are Daydream-certified — a side effect of the high standard of experience that Google is hoping to maintain — but you can bet that many Android phones will support the standard in 2017.

VR, AR, MR and every other “R” need to coexist for a while. For now virtual reality is the easiest to pull off — software and hardware makers have the fewest things to keep track of and complete control of the virtual environment — and also the most developed. It’s fairly easy for a developer to build a VR app or for a manufacturer to make a VR-ready phone. Mixed reality is clearly harder.

Microsoft’s HoloLens is effectively a wearable computer, making thousands of calculations every second just to understand its environment. And its limitations, such as field of view, are way more apparent than those of a VR headset. The virtual objects of HoloLens have to be small enough — or faraway enough — to fit into a small square in the middle of the headset. You simply can’t see the whole illusion. Perhaps Magic Leap already has the answer to that problem, but given how many years it’s been in development — and how little it’s shown so far — it’s likely not a simple thing to figure out.

In 2017, Microsoft’s partners will release a handful of $300 VR headsets for Windows. Rather than competing with existing VR products, these headsets are more like a diet HoloLens. You’ll get the same experience, interface and apps as HoloLens, but your entire environment will be virtual. Think of it like a gateway drug for mixed reality. In one swoop, it’s getting both developers and users ready for MR, without the tribulations of dealing with first-generation, hyper-expensive headsets.

At the same time, Google is currently working on a device that uses cameras and algorithms to display mixed reality inside a virtual reality headset. It’s essentially going to be a combination of VR and Google’s Tango computer vision efforts, with a lot of extra smarts added on top. Again, the project seems almost like a stepping-stone toward a more complete mixed reality experience. The device has yet to be announced, but sources familiar with the matter say it’s of great importance to the company.

The dark horse in all of this is Apple. As is tradition, there’s been a lot of speculation and questions asked about the company’s plans for virtual, augmented and mixed reality. CEO Tim Cook has said that AR is more interesting than VR, as it’s less closed off and more social. The company has already acquired an AR company, and it has experts in the field within its ranks. Its iPhones clearly have the power and sensors to pull off a Daydream-like VR experience immediately, but it’s obviously waiting to offer something more compelling to its users.

There can be no doubt that ‘virtual reality’ headsets like the Vive, Rift and Daydream View are just a stopgap until mixed reality is ready.

There can be no doubt that “virtual reality” headsets like the Vive, Rift and Daydream View are just a stop-gap until mixed reality is ready. That probably sounds like a bold statement, but it’s easy to justify. Mixed reality headsets will, at some point make virtual objects appear solid. HoloLens isn’t there yet, sure, but Magic Leap claims to be, and you can be sure Microsoft is working on it.

Once these headsets are able to display opaque objects and cover our entire field of view, developers and creatives will have total control over what we see. They can decide to mix or augment our surroundings, like we’ve already seen with Magic Leap and HoloLens, or completely scrap that environment and put us in a virtual space, like with a VR headset. It should only take a few taps to send us to an augmented reality, a virtual one and back to our own.

This year showed millions of people how fun it can be to see a digital creation entering their world. And maybe 2017 won’t be the year, but as technology catches up to its aspirations, we might soon be able to see how fun it is to have millions of digital creations do the same.

Check out all of Engadget’s year-in-review coverage right here.