Ukraine’s military wants HoloLens helmets for its tank commanders

It’s not easy to see out of a tank (that’s the point) but in order to be effective on the battlefield, their crews need to know what’s going on around them. Modern tanks often have a variety cameras mounted to their exteriors to help the soldiers inside get a better view but crews still have to rely on monitors in the cabin to see out. However, a new HoloLens-enabled helmet from Limpid Armor can give tank commanders a better view of their surroundings just by turning their heads.

The helmet, dubbed the Circular Review System (CRS), sets a HoloLens headset directly onto the front of its frame. Video feeds gathered from the tank’s exterior cameras are stitched together and displayed through the headset as a “mixed reality” view to the wearer. With it, tank crews are afforded a 360-degree view of the situation in both the visible and infrared spectrums. Not only that, the CRS can tag enemy and friendly soldiers, designate targets and feed critical information to the commander.

Limpid Armor debuted the Limpid Armor in mid-October at the Arms and Security show held in Kiev. The company has not yet tested the helmet outside the lab, however, the Ukraine military has expressed interest in the technology so, given the region’s heightened tensions surrounding Russia’s illegal annexation of the Crimean peninsula, expect for field tests to come quickly.

Via: VR Scout

Source: Limpid Armor (Facebook)

3D audio is the secret to HoloLens’ convincing holograms

The streets of Microsoft’s campus are lined with tall fir trees. A drive through lush, green urban woods reveals dozens of nondescript buildings. Minibuses shuttle employees across the company’s 500-acre headquarters in Redmond, Washington. Inside Building 99, a concrete-and-glass structure that houses Microsoft Research, Ivan Tashev walked through the quiet halls toward his lab, where he devised the spatial sound system for HoloLens.

Tashev leads the audio group at Microsoft Research, which is the second largest computer science organization in the world. For HoloLens, a mixed-reality headset that places holograms in your immediate environment, his team devised a sound system that creates the illusion of 3D audio to bring virtual objects to life.

Mixed reality, like virtual reality, is a medium best known for its visual trickery. When you first try on the HoloLens, the thing that instantly grabs your attention is the holographic display: the aliens crawling out of the walls in RoboRaid or Buzz Aldrin walking on the surface of Mars. The device tricks your brain into seeing things that are only visible through the headset. But what makes the holograms seem realistic is the spatial sound system that allows you to engage with the projections. You hear the alien enemies before they break out of the walls, and you can find the astronaut talking to you as he walks across the red planet.

“Spatial sound roots holograms in your world,” says Matthew Lee Johnston, audio innovation director at Microsoft. “The more realistic we can make that hologram sound in your environment the more your brain is going to interpret that hologram as being in your environment.”

The HoloLens audio system replicates the way the human brain processes sounds. “[Spatial sound] is what we experience on a daily basis,” says Johnston. “We’re always listening and locating sounds around us; our brains are constantly interpreting and processing sounds through our ears and positioning those sounds in the world around us.”

The brain relies on a set of aural cues to locate a sound source with precision. If you’re standing on the street, for instance, you would spot an oncoming bus on your right based on the way its sound reaches your ears. It would enter the ear closest to the vehicle a little quicker than the one away from it. It would also be louder in one ear than the other based on proximity. These cues help you pinpoint the location of things. But there’s another physical factor that impacts the way sounds are perceived.

Before a sound wave enters a person’s ear canals, it interacts with the outer ears, the head and even the neck. The shape, size and position of the human anatomy add a unique imprint to each sound. The effect, which is called Head-Related Transfer Function (HRTF), makes everyone hear sounds a little differently.

These subtle differences make up the most crucial part of a spatial sound experience. For the aural illusion to work, all the cues need to be generated with precision. “A one-size-fits-all [solution] or some kind of generic filter does not satisfy around one-half of the population of the Earth,” says Tashev. “For the [mixed-reality experience to work], we had to find a way to generate your personal hearing.”

His team started by collecting reams of data in the Microsoft Research Lab. They captured the HRTFs of hundreds of people to build their aural profiles. The acoustic measurements, coupled with precise 3D scans of the subjects’ heads, collectively built a wide range of options for the HoloLens. A quick and discreet calibration can match the spatial hearing of the device user to the profile that comes closest to his or hers.

Drone concept for RoboRaid. Image: Microsoft

Back at the Microsoft campus, on a bright, sunny morning in late August, Tashev walked into his lab at Building 99. Dressed in black pants and a platinum gray shirt that matched his hair, he pulled open the heavy doors to a concealed room where he carries out the acoustic measurements. The walls, covered with large foam wedges, insulate the space from the rest of the building. The floor is made up of a wire mesh that sits atop another layer of sound absorbers at the bottom. The structure soaks up all sounds and vibrations to create an anechoic chamber, or a space that is devoid of echoes.

The Harvard anechoic chamber built in 1943. Image: Harvard University Archives

After a few minutes, the echoless chamber starts to feel uncomfortable, even unnatural. The blood pumping through the heart becomes more audible. The ebb and flow of the air in the lungs comes into focus. It’s a feeling that is often experienced inside anechoic rooms, which have been around for many decades. Dr. Leo Beranek, the director of Harvard’s electroacoustic lab, built the first one in 1943 to test broadcasting systems and loudspeakers and to improve noise control during WWII. Since then, similar spaces have been designed to test microphones and to measure HRTFs for multi-directional audio systems.

At Microsoft Research, Tashev’s chamber has a black leather chair at the center of the room where the HRTFs of 350 people have been measured. After a pair of small orange microphones has been placed inside the ears of a subject, a black rig, equipped with 60 speakers, slowly rises from the back. As the contraption moves in an arc over the person, it stops at brief intervals to play sharp, successive laserlike sounds. The microphones capture the sound waves as they enter the ear canals of the participant.

By playing sounds all around the listener, the team is able to capture the precise audio cues for both right and left ears in relation to 400 directions in the room. These measurements give them a pair of HRTF filters for each sound source. “If we know these filters for all possible directions, then we own your spatial hearing,” says Tashev. “We can trick your brain and make you perceive that the sound comes from any desired direction.”

Your browser does not support the audio element

Hit play to hear the sounds Microsoft Research used for acoustical measurements.

To place a hologram at a particular location, a corresponding audio filter is applied. When the HoloLens projects those specific sounds, the HRTF clues baked into them trick the human brain into spotting the source almost instantly.

Despite the realism, the paraphernalia required to generate spatial sound has kept it from replacing stereo and surround systems for the masses. Apart from the precise acoustic measurements, it also requires constant head-tracking. The orientation of the head has a direct impact on the way sounds reach the ears. If you’re looking away from the bus on the street, for instance, it will sound different than if you’re looking straight at it.

For HoloLens, however, the team did not need to tackle the head-tracking problem from scratch. The holographic visuals work in part because one of the six cameras in the device monitors the user’s head movements at all times. The audio system simply taps into that information.

Microsoft is not the first or only company with the ability to create personalized audio. For most 3D audio experiences in VR, creators have been relying on HRTF databases that are publicly available or turning to research labs where audio personalization has been possible for a number of years. At Princeton University, Edgar Choueiri, a professor of mechanical and aerospace engineering, has been using the microphone-in-ears technique for the past few years. And VisiSonics, a company based in the University of Maryland’s research lab, has been measuring HRTFs to build its own library.

But Microsoft’s audio system stands apart for its engineering, which makes the audio calibration invisible to the HoloLens user. While the personalization isn’t as perfect as it tends to be inside a controlled lab, it is a lot less tedious.

The first time you wear the device, you start with a wizard that guides you through a calibration for the eyes. For the holographic effect to work, the computer around your head needs to measure the distance between your pupils. It asks you to close one eye, hold your finger up and tap down on a projected image in front of you. You repeat the same for the second eye for the system to calculate the interpupillary distance. But that’s not all the system is doing. Baked into this process is an algorithm that correlates the eye measurements with the numbers from Tashev’s research that scanned and measured the eyes and ears of hundreds of subjects to build a generic average. Essentially, the distance between the eyes becomes an indicator of the distance between the two ear canals of the person using the device.

The idea is to make the information-gathering process as inconspicuous as possible. “I think we succeeded because today the final user doesn’t even know when or how the personalization of the HRTFs happens,” says Tashev. “We made it transparent for the user.”

The efficiency of software also extends to the hardware in the HoloLens. While the dynamism of spatial sound is best maintained and experienced over headphones, the HoloLens team needed to steer clear of any occlusions to keep the mixed-reality effects intact. “We quickly realized that the user would like to hear the environment around them in addition to the sound from the holograms,” says Hakon Strande, senior program manager at Microsoft. “So we needed something that was outside the ear but close enough to make sure the sound reached the ear at a certain level of loudness.”

Strande describes one early iteration of the HoloLens that had small tubes folding down from the band to direct air into the ear canals. Another concept swapped the tubes for earbuds that popped into the ears of the user. But the team eventually engineered a pair of thin, red speakers that sit on a band right above the user’s ears.

“Most people don’t realize that [the speakers] are there,” says Strande. “The first time they try the device and when they hear the sound in the space around them they think there are speakers located in the room around them that are playing the sounds. That’s how convincing the effect and simulation is at this point.”

Microsoft’s spatial audio, while active and effective in the HoloLens, isn’t limited to the device. It’s essentially baked into the operating system so it can work across devices that rely on Windows 10. With new VR headsets announced for the operating system at the Surface event in October, perhaps the spatial-audio technique will translate from a holographic mixed reality to a fully immersive virtual space.

“Audio is important in mixed reality and in VR because it ties the experience together,” says Strande. “It is often the second thing that game and app developers think about but without audio you don’t suspend disbelief. To bring something to life, it has to have a sound aspect to it — especially if they’re holograms that are moving around you.”

Lenovo, HP and Dell are all working on $300 Windows VR headsets

Microsoft’s holographic computing platform will crossover to the world of virtual reality. It was first announced at Computex earlier this year and today at the company’s Surface event in New York City that vision came to life on stage. The new Windows Creators update will fire up VR headsets from companies like HP, ASUS, Lenovo, Dell and Acer. Modeled after Microsoft’s HoloLens, these new headsets will carry an internal tracking system and will also be untethered. The devices will work with Windows PCs and will start at $299.

Developing…

Click here to catch all the latest news from Microsoft’s big Surface event.

Microsoft’s HoloLens is now available to pre-order in Europe

With all of the hype around VR, it’s easy to forget Microsoft’s push into augmented reality (AR). Today, the company is launching HoloLens pre-orders in a bunch of new countries, including Australia, France, Germany, Ireland, New Zealand and the UK. In (not so) good old British Sterling, the headset will set you back an eye-watering £2,719, while the enterprise-ready “Commercial Suite” version costs £4,529. Microsoft says the first units will ship in late November — that way, you can wow your friends and family with some tabletop-projected Minecraft over Christmas.

HoloLens has been a slow burn for Microsoft. The company kicked off pre-orders in the US and Canada on February 29th, with the first deliveries starting a month later. The use-cases are massive, but the high cost of entry has limited the hardware to deep-pocketed developers. That could change, however, if Microsoft develops HoloLens as a platform. At Computex in June, the company opened up its “Windows Holographic” initiative to third parties, enabling devices that can run both AR and VR “mixed reality.” Similar to Google Daydream, this could kick-start a wider ecosystem of HoloLens-style headsets, increasing sales and developer interest.

Duke hopes HoloLens will assist with tricky brain surgeries

When Microsoft envisioned the medical world embracing HoloLens in the future, it wasn’t kidding around. Duke University is testing the augmented reality headset as an assistive tool for difficult brain surgeries like extraventricular drain placement, which relieves potentially fatal pressure. Instead of relying on CT scans and markers to insert a catheter into the skull during the draining procedure, Duke’s doctors would use HoloLens to overlay a reconstructed CT scan on the patient’s head. The virtual approach should not only be more accurate than conventional markers (the target is frequently too small or shifts around), but faster and simpler.

The newer hardware is particularly important, too. Augmented reality surgery is nothing new, but HoloLens eliminates a large amount of the jittering and lag that simply wouldn’t be acceptable during delicate operations.

Duke has only tested HoloLens on a model so far, and only with that one procedure. It won’t shock you to hear that the school wants to prove the wearable can work in real situations and different procedures before there’s widespread adoption. However, it’s not hard to see HoloLens and other augmented reality gear becoming a mainstay in operating theaters. Surgeons could spend more time completing procedures and less time fretting that one slip might endanger a patient.

Via: TechCrunch

Source: Duke University

Microsoft launches Actiongram on HoloLens with a tiny George Takei

Ever wanted to hang out with a tiny, goofball celebrity? If you happen to have a HoloLens augmented reality headset, now you can. Today Microsoft made Actiongram, its mixed-reality filmmaking app, available to all HoloLens owners — and celebrated the launch by giving users access to a hologram of George Takei.

The tiny Takei character is pretty adorable, and allows users to superimpose a laughing, tickling and sword wielding hologram of the actor on any video filmed with the augmented reality headset. The public beta also comes with themed packs with Nyan Cat, Grumpy Cat, dinosaurs, unicorns, historical figures and more. The idea is to allow users to add “hollywood” effects to socially sharable videos without any special skills or expertise — but so far, it’s only been available to a select group of “social storytellers.” Today, that elite group narrows: now all you need is $3,000 to buy a Hololens developer headeset.

Source: Twitter

HoloLens’ 24-core chip makes vivid AR possible

Microsoft has revealed more details about its HoloLens headset at the Hot Chips conference in Cupertino. The augmented reality (AR) device marries video games with the real world, so it needs to calculate your location, head position and gestures extremely rapidly. We now know more about the so-called holographic processing unit (HPU) that does that — the 24-core chip has near-supercomputer levels of performance, performing a trillion operations per second.

The 28-nanometer process “Tensilica” chip, built by TSMC, has 65 million logic gates, 8MB of ultra-fast static RAM (SRAM) and 1GB of DDR3 RAM, all crammed into a 12mm x 12mm (half-inch square) package. It’s designed to process input from the inertial chip, environment cameras, depth camera and other sensors. By speeding up calculations 200 times over software alone, it provides ready-to-use position data to the headset’s main Intel “Cherry Trail” CPU, freeing it to run apps and games.

If you live in the US or Canada and have $3,000 between your couch cushions, you can actually buy a HoloLens developer model. If not, we tested it recently and found it fun, immersive and not nearly as disorienting (or nausea-inducing) as a VR headset. It’s very hard to say how much the final product will cost, considering that it has a highly custom chip on board. We’ll likely have to wait a while to find out, though, as there’s no indication yet on when the retail version will arrive.

Source: The Register

Microsoft gives everyone a chance to buy a HoloLens

In the past, developers had to apply for a chance to buy Microsoft’s augmented reality headset. Now, the tech titan has loosened up its requirements for purchase. You only have to meet three conditions to be able to get up to five HoloLens units: 1.) you have a Microsoft account, 2.) you have an address in the US or Canada, and 3.) you can afford the device’s $3,000 price tag. The device is officially on offer for developers, but so long as all three requirements aren’t an issue, you can get one (or five) straight from the Microsoft store. We’ll bet the consumer edition will be easier on your wallet, though. So if you’re not developing apps for the device, you can always wait a bit longer to make sure you can afford rent next month.

Via: TechCrunch

Source: Microsoft