The brain hacks that make climbing in VR feel real

When I talk to friends and family about VR, their most pressing questions are usually about immersion. Once they’ve finished asking about the possibility of vomiting, the conversation turns to: “And how real does it feel?” “Do you believe you’re really there?” Truth be told, I’ve never had that sensation — a complete and utter submission of my senses — although developers are getting better at tricking my brain for a few fleeting moments.

Take two VR climbing projects that are currently in development: The Climb and Everest VR. The former, an Oculus Rift game by Crytek, duped my body into sweating at a few crucial checkpoints peppered throughout the cliff face. The latter, which takes you to some of the most iconic and dangerous parts of Mount Everest, triggered a surprising sense of vertigo as I walked nervously across an icy crevasse.

The two experiences use different techniques to lead the player into feeling certain sensations. The reasons are numerous. For one, the creators are conveying different places and activities, which in turn have different emotions attached to them. For another, The Climb is being pitched as a straight-up video game, albeit with simple controls, whereas Everest VR is more of a cinematic tour with minimal challenge and exploration.

When I dive into Crytek’s rock-climbing romp, I start halfway up a rock face in Halong Bay, Vietnam. Within a few seconds, I’m dangling from a tiny hand hold, peering down at the glassy ocean hundreds of feet below. It’s a breathtaking view and when I turn around, I see that my body is represented by two dismembered hands, which can be clenched using the left and right triggers on the gamepad. When you release one, it’ll float in mid-air and move as you look around with the headset, finally hovering over a hand hold if it’s within your reach. If you press the trigger again, it’ll suddenly snap into place and cause your body to move upwards.

Such a control scheme might sound a little strange, but it’s surprisingly natural in practice. I’ve been bouldering a few times (rock climbing, but without the ropes) and have become accustomed to dangling with one hand, looking around a corner and then delicately reaching with my free arm. You quickly get into a rhythm — a methodical left, right, left, right — and that feeling of momentum is replicated in The Climb, which is impressive given that most of your body is strangely invisible.

Stressing the risks

These controls are merely the foundation that cement the feeling of climbing, however. When I found myself sweating at certain checkpoints, it was for two reasons. Firstly, a small sense of physical exhaustion after completing what would be a tremendously long and difficult climb in the real world. Secondly, and more overwhelmingly, was a sense of relief. The latter, I believe, was triggered because of the technicality of the climb — like a long video game boss battle that takes every ounce of your concentration — as well as the inherent dangers of climbing.

The game achieves this sense of tension and risk-taking with three buttons on the controller: a face-button for jumping and two bumpers for reapplying climbing chalk. The chalk acts as two stamina gauges, one for each hand, which slowly deplete as you shimmy around corners and scramble up ledges. The more complex the maneuver, the bigger the toll on your climbing chalk. Your ghostly hands will change from white to red and so, before tackling a particularly difficult section, you’ll want to stop and prepare by reapplying the fine, sweat-repelling powder.

Jumping is a huge gamble. You have to judge the distance, leap and then hit the triggers at just the right moment to safely grip the hand hold. On multiple occasions I missed, swearing profusely as I plummeted to my inevitable death. As a beginner, this sequence can be a little frightening. But for experienced players, it’ll soon be replaced by a feeling of frustration, given the challenge and replayability comes with completing the climbs in faster times.

A death-defying leap will also eat into your chalk, forcing you to stop and recover. It’s a small, but subtle technique that reinforces the sense of exertion and the physical penalties involved with rock climbing. Once I reached the top and took off the headset, I wanted to sit down and catch my breath for a moment.

Scaling Mount Everest

Everest VR, which is being developed for the HTC Vive, is taking a different approach. Sólfar Studios, a developer of VR experiences in Reykjavik, Iceland, is working with the visual effects studio RVX on a series of linked vignettes. The one I tried took place on the Khumbu Icefall, a dangerous section where crevasses can open at any moment. For this particular demo, I was wearing an absurdly large jacket in a room with “snow” on the floor and flags with Tibetan script hanging from the ceiling. The air conditioning had also been lowered and while I couldn’t see my breath, it did feel just a teensy-bit more like Everest. Not that Sólfar expects you to go to these lengths in your living room, but every little bit helps.

After a brief cinematic, I’m asked to step on a set of footprints in the corner of the room. Once I’ve found them with my goggles, Mount Everest suddenly appears around me, with a narrow ladder stretching over a deadly chasm. Immediately, the detail and authenticity of the environment is apparent. Unlike The Climb, which is based loosely on a real world location, Everest VR is aiming for absolute accuracy.

Initially, RVX was working on a model of the mountain for a feature film, which is also called Everest and came out earlier this year. “I thought it was very important to be completely accurate, in terms of all the geography, the topology, and the different views from different places,” Dadi Einarsson, RVX’s Creative Director says. The company used a technique called photogrammetry, which involves taking photographs from numerous vantage points to record and construct a three-dimensional surface. This, combined with “a huge mish-mash of different sources,” as Einarsson describes it, piqued the curiosity of Sólfar Studios.

“We immediately thought it would be extremely cool to bring this to VR,” Kjartan Emilsson, CEO of Sólfar Studios recalls. “With the level of detail that was there, we knew we would be able to create a sense of immersion.”

The result is impressive. I tentatively crept forward and pressed the triggers on two wand-shaped controllers, forcing my virtual mitts to bind myself to a rope system. I slowly inch across the ladder until, halfway across, I stop and take a long look down. To my great surprise, this managed to create a brief sense of vertigo. I could feel my stomach tightening and my legs turning to putty. It quickly passed, however, and after gathering my thoughts I shuffled across to the far side.

That phantom sense of vertical giddiness was triggered purely by the quality of the environment. Which is impressive, given it’s a digital reconstruction of the mountain, not a 360-degree video.

Sólfar Studios says it’s working on other ways to “hack the brain” during its Everest sequences. For instance, when you enter the “death zone,” which climbers use to describe an altitude where there isn’t enough oxygen to breath properly, the team wants players to move in a slow, deliberate manner. But that’s difficult, given there’s nothing in the room to physically slow you down. Sólfar’s solution is to blur your vision, as if you were blacking out, whenever you move too fast. In addition, there will be a subtle but deliberate audio track in the background imitating a heartbeat. Even if you don’t notice it, the company says your body should naturally align with it and discretely emphasize the difficult conditions.

These techniques are mostly experimental. Some could be trialled in a traditional video game, but others feel unique to VR. In the final version of The Climb, for instance, you’ll be able to use the Oculus Touch controllers to reach out and grab parts of the cliff face. Technically, this could have been possible with other motion-based controllers, like the PlayStation Move, but it’s the culmination of the Rift’s hardware pieces — the headset, the Touch controllers and a decent set of over-ear headphones, that is giving Crytek new ways to manipulate our senses.

The same is true of Everest VR. The “Lighthouse” tracking system that comes with the HTC Vive offers accurate motion tracking that is, in my opinion, far superior to Microsoft’s first and second-gen Kinect peripherals. The various vignettes could have been offered on a console or a high-powered gaming PC, but it wouldn’t have had the same effect. With a VR headset, I can look down and see my hands as snow-covered gloves, rather than pasty fingers wrapped around two plastic controllers. That sense of immersion is what allows Sólfar Studios and RVX to play with the body’s expectations in ways that would have been impossible, or felt contrived, with a normal TV and speaker setup.

If VR can hack my brain, even for a moment, it bodes well for the medium’s future. It’s unlikely that I’ll ever scale Everest in the real world, but I look forward to the day when I can pull on a pair of goggles and truly believe that I’m standing at the summit of the world’s tallest mountain.