3D audio is the secret to HoloLens’ convincing holograms
The streets of Microsoft’s campus are lined with tall fir trees. A drive through lush, green urban woods reveals dozens of nondescript buildings. Minibuses shuttle employees across the company’s 500-acre headquarters in Redmond, Washington. Inside Building 99, a concrete-and-glass structure that houses Microsoft Research, Ivan Tashev walked through the quiet halls toward his lab, where he devised the spatial sound system for HoloLens.
Tashev leads the audio group at Microsoft Research, which is the second largest computer science organization in the world. For HoloLens, a mixed-reality headset that places holograms in your immediate environment, his team devised a sound system that creates the illusion of 3D audio to bring virtual objects to life.
Mixed reality, like virtual reality, is a medium best known for its visual trickery. When you first try on the HoloLens, the thing that instantly grabs your attention is the holographic display: the aliens crawling out of the walls in RoboRaid or Buzz Aldrin walking on the surface of Mars. The device tricks your brain into seeing things that are only visible through the headset. But what makes the holograms seem realistic is the spatial sound system that allows you to engage with the projections. You hear the alien enemies before they break out of the walls, and you can find the astronaut talking to you as he walks across the red planet.
“Spatial sound roots holograms in your world,” says Matthew Lee Johnston, audio innovation director at Microsoft. “The more realistic we can make that hologram sound in your environment the more your brain is going to interpret that hologram as being in your environment.”
The HoloLens audio system replicates the way the human brain processes sounds. “[Spatial sound] is what we experience on a daily basis,” says Johnston. “We’re always listening and locating sounds around us; our brains are constantly interpreting and processing sounds through our ears and positioning those sounds in the world around us.”
The brain relies on a set of aural cues to locate a sound source with precision. If you’re standing on the street, for instance, you would spot an oncoming bus on your right based on the way its sound reaches your ears. It would enter the ear closest to the vehicle a little quicker than the one away from it. It would also be louder in one ear than the other based on proximity. These cues help you pinpoint the location of things. But there’s another physical factor that impacts the way sounds are perceived.
Before a sound wave enters a person’s ear canals, it interacts with the outer ears, the head and even the neck. The shape, size and position of the human anatomy add a unique imprint to each sound. The effect, which is called Head-Related Transfer Function (HRTF), makes everyone hear sounds a little differently.
These subtle differences make up the most crucial part of a spatial sound experience. For the aural illusion to work, all the cues need to be generated with precision. “A one-size-fits-all [solution] or some kind of generic filter does not satisfy around one-half of the population of the Earth,” says Tashev. “For the [mixed-reality experience to work], we had to find a way to generate your personal hearing.”
His team started by collecting reams of data in the Microsoft Research Lab. They captured the HRTFs of hundreds of people to build their aural profiles. The acoustic measurements, coupled with precise 3D scans of the subjects’ heads, collectively built a wide range of options for the HoloLens. A quick and discreet calibration can match the spatial hearing of the device user to the profile that comes closest to his or hers.
Drone concept for RoboRaid. Image: Microsoft
Back at the Microsoft campus, on a bright, sunny morning in late August, Tashev walked into his lab at Building 99. Dressed in black pants and a platinum gray shirt that matched his hair, he pulled open the heavy doors to a concealed room where he carries out the acoustic measurements. The walls, covered with large foam wedges, insulate the space from the rest of the building. The floor is made up of a wire mesh that sits atop another layer of sound absorbers at the bottom. The structure soaks up all sounds and vibrations to create an anechoic chamber, or a space that is devoid of echoes.
The Harvard anechoic chamber built in 1943. Image: Harvard University Archives
After a few minutes, the echoless chamber starts to feel uncomfortable, even unnatural. The blood pumping through the heart becomes more audible. The ebb and flow of the air in the lungs comes into focus. It’s a feeling that is often experienced inside anechoic rooms, which have been around for many decades. Dr. Leo Beranek, the director of Harvard’s electroacoustic lab, built the first one in 1943 to test broadcasting systems and loudspeakers and to improve noise control during WWII. Since then, similar spaces have been designed to test microphones and to measure HRTFs for multi-directional audio systems.
At Microsoft Research, Tashev’s chamber has a black leather chair at the center of the room where the HRTFs of 350 people have been measured. After a pair of small orange microphones has been placed inside the ears of a subject, a black rig, equipped with 60 speakers, slowly rises from the back. As the contraption moves in an arc over the person, it stops at brief intervals to play sharp, successive laserlike sounds. The microphones capture the sound waves as they enter the ear canals of the participant.
By playing sounds all around the listener, the team is able to capture the precise audio cues for both right and left ears in relation to 400 directions in the room. These measurements give them a pair of HRTF filters for each sound source. “If we know these filters for all possible directions, then we own your spatial hearing,” says Tashev. “We can trick your brain and make you perceive that the sound comes from any desired direction.”
Your browser does not support the audio element
Hit play to hear the sounds Microsoft Research used for acoustical measurements.
To place a hologram at a particular location, a corresponding audio filter is applied. When the HoloLens projects those specific sounds, the HRTF clues baked into them trick the human brain into spotting the source almost instantly.
Despite the realism, the paraphernalia required to generate spatial sound has kept it from replacing stereo and surround systems for the masses. Apart from the precise acoustic measurements, it also requires constant head-tracking. The orientation of the head has a direct impact on the way sounds reach the ears. If you’re looking away from the bus on the street, for instance, it will sound different than if you’re looking straight at it.
For HoloLens, however, the team did not need to tackle the head-tracking problem from scratch. The holographic visuals work in part because one of the six cameras in the device monitors the user’s head movements at all times. The audio system simply taps into that information.
Microsoft is not the first or only company with the ability to create personalized audio. For most 3D audio experiences in VR, creators have been relying on HRTF databases that are publicly available or turning to research labs where audio personalization has been possible for a number of years. At Princeton University, Edgar Choueiri, a professor of mechanical and aerospace engineering, has been using the microphone-in-ears technique for the past few years. And VisiSonics, a company based in the University of Maryland’s research lab, has been measuring HRTFs to build its own library.
But Microsoft’s audio system stands apart for its engineering, which makes the audio calibration invisible to the HoloLens user. While the personalization isn’t as perfect as it tends to be inside a controlled lab, it is a lot less tedious.
The first time you wear the device, you start with a wizard that guides you through a calibration for the eyes. For the holographic effect to work, the computer around your head needs to measure the distance between your pupils. It asks you to close one eye, hold your finger up and tap down on a projected image in front of you. You repeat the same for the second eye for the system to calculate the interpupillary distance. But that’s not all the system is doing. Baked into this process is an algorithm that correlates the eye measurements with the numbers from Tashev’s research that scanned and measured the eyes and ears of hundreds of subjects to build a generic average. Essentially, the distance between the eyes becomes an indicator of the distance between the two ear canals of the person using the device.
The idea is to make the information-gathering process as inconspicuous as possible. “I think we succeeded because today the final user doesn’t even know when or how the personalization of the HRTFs happens,” says Tashev. “We made it transparent for the user.”
The efficiency of software also extends to the hardware in the HoloLens. While the dynamism of spatial sound is best maintained and experienced over headphones, the HoloLens team needed to steer clear of any occlusions to keep the mixed-reality effects intact. “We quickly realized that the user would like to hear the environment around them in addition to the sound from the holograms,” says Hakon Strande, senior program manager at Microsoft. “So we needed something that was outside the ear but close enough to make sure the sound reached the ear at a certain level of loudness.”
Strande describes one early iteration of the HoloLens that had small tubes folding down from the band to direct air into the ear canals. Another concept swapped the tubes for earbuds that popped into the ears of the user. But the team eventually engineered a pair of thin, red speakers that sit on a band right above the user’s ears.
“Most people don’t realize that [the speakers] are there,” says Strande. “The first time they try the device and when they hear the sound in the space around them they think there are speakers located in the room around them that are playing the sounds. That’s how convincing the effect and simulation is at this point.”
Microsoft’s spatial audio, while active and effective in the HoloLens, isn’t limited to the device. It’s essentially baked into the operating system so it can work across devices that rely on Windows 10. With new VR headsets announced for the operating system at the Surface event in October, perhaps the spatial-audio technique will translate from a holographic mixed reality to a fully immersive virtual space.
“Audio is important in mixed reality and in VR because it ties the experience together,” says Strande. “It is often the second thing that game and app developers think about but without audio you don’t suspend disbelief. To bring something to life, it has to have a sound aspect to it — especially if they’re holograms that are moving around you.”
Gawker settles with Hulk Hogan for a reported $31 million
Since Gawker lost its court battle with Hulk Hogan back in June and filed for bankruptcy, it’s been unclear just how much the former professional wrestler would get given the massive $140 million judgment awarded him. But today, the media company’s cofounder and CEO Nick Denton announced that they’ve settled with the celebrity for $31 million.
In a blog post, Denton explained the rationale behind the lower number: In short, everyone needs to move on. The four-year legal battle stretched resources, such that appealing both the Hogan case and his lawyer Charles Harder’s other two cases involving Gawker stories would exhaust more time and money. As part of the settlement, all three posts have been taken down, which Denton called “the most unpalatable part of the deal.”
They didn’t pursue the appeals process due to the large financial elephant in the room: Peter Thiel. The tech billionaire had bankrolled Harder to take on Hogan’s case, spending his own money to avenge Gawker’s outing him as gay in 2007. Denton was convinced that Thiel would continue dragging the fight out and even likely fund other cases if the media company won its appeal, as the billionaire said he would do in a New York Times op ed.
But the other big reason to settle the four-year case: Everyone needs to move on. Instead of extending the litigation, Hogan will retire comfortably and the article’s writer A.J. Daulerio, who was also named a defendant in the case, will be able to proceed. Shed of the Gawker site and name, the newly-branded Gizmodo network (including Jalopnik, Jezebel, Kotaku, Lifehacker and Deadspin) can forge ahead under ownership by Univision.
So the saga is over, as Denton wrote in his post. All the writers involved, including him, can move on. But the specter of Thiel’s then-secret machinations to bring down a media company that crossed him remain. The billionaire appointed himself a First Amendment gatekeeper: At a National Press Club Q&A on Monday, Thiel told reporters that Gawker’s employees were not journalists and didn’t deserve Constitutionally-bound right of freedom of the press.
Except, they do, because the law of the land says so. Thus Thiel hit them where he could, in the pocketbook, bankrolling court cases from the shadows because, in his esteem, Hogan was too poor to prevail in the court system alone. As he told the National Press Club: “If you’re middle-class, if you’re upper-middle class, if you’re a single-digit millionaire like Hulk Hogan, you have no effective access to our legal system. It costs too much.”
For Thiel, an investor in Facebook and Palantir, the Hogan case probably cost him less than 1% of his net worth to fund, Denton estimated. Perhaps that is the lasting legacy of this whole sad debacle. As he concludes in his post:
“It’s a shame the Hogan trial took place without the motives of the plaintiff’s backer being known. If there is a lasting legacy from this experience, it should be a new awareness of the danger of dark money in litigation finance. And that’s surely in the spirit of the transparency Gawker was founded to promote. As for Peter Thiel himself, he is now for a wider group of people to contemplate.”
Via: CNBC
Source: Nick Denton
An artwork controlled by a colony of bacteria
The Tate Modern’s Turbine Hall has always been a vacuous space. Five storys high, with 35,000 sq ft. of space for artworks, it’s been home to some of the London museum’s most memorable exhibitions. Its latest, by sheer spirit of invention, is no exception.
Anywhen is an immersive artwork by French avant-garde artist Philippe Parreno. Billed as an exhibition that challenges your perception of time and space, it’s essentially, as The New Scientist puts it, a “factory-sized [children’s] mobile.” It features an ever-changing mix of sound, light and shadow, augmented by fish-shaped balloons, a transforming array of suspended speakers and a cinema that seemingly appears at random.
Except, it’s not random. The individual elements of Anywhen are at the whim of bacteria. Bioreactors sit in the corner of the Turbine Hall, housing a colony of yeast cultures. Data on the colony’s movements, temperature and growth is being collected by scientists from University College London, and is fed into an algorithm that controls the artwork’s many elements. As the cultures mature over the course of the exhibition, they’ll forever change the patterns that play out. The artwork a visitor to the Tate Modern will see today will not be the artwork on display next year.
This isn’t the first time that Parreno has experimented with bacteria for art. Back in April he collaborated with Barbara Gladstone for a New York show that also utilized bioreactors. Speaking to The Art Newspaper, Parreno explained that he “liked the idea that this colony of bacteria can actually control your surroundings and affect the huge space of the Turbine Hall.” While he created each element, the moments that visitors will experience are being dictated by the colony. “The bacteria that control your surroundings by responding to information coming from its surroundings.”
The Big Picture is a recurring feature highlighting beautiful images that tell big stories. We explore topics as large as our planet, or as small as a single life, as affected by or seen through the lens of technology.
Microsoft Debuts ‘Teams’ Chat-Based Workspace and Slack Competitor
Microsoft today debuted its latest product, Microsoft Teams, which is a chat-based workspace designed for Office 365 users.
Designed to compete with chat platforms like Slack and HipChat, Microsoft Teams provides a chat interface that integrates with Office 365 apps and services and other third-party services like Zendesk, Asana, Hootsuite, and Intercom.
According to Microsoft, Teams is designed to provide a “modern conversation experience” in the workplace. It supports both persistent and threaded chats, along with public and private conversations. Skype integration allows teams to quickly initiate voice and video conferences, and each digital workspace can be highly customized with emoji, stickers, GIFs, extensions, open APIs, and more.
At Microsoft, we are deeply committed to the mission of helping people and organizations achieve more–and reinventing productivity for the cloud and mobile world is core to our ambition. We built Microsoft Teams because we see both tremendous opportunity and tremendous change in how people and teams get work done.
Teams are now more agile and organizational structures more flat to keep communications and information flowing. With Microsoft Teams, we aspire to create a more open, digital environment that makes work visible, integrated and accessible–across the team–so everyone can stay in the know.
Word, Excel, PowerPoint, SharePoint, OneNote, Planner, Power BI, and Delve are built into Microsoft Teams, and it supports Microsoft’s cross-application membership program, Office 365 Groups, so people can easily move from conversations to collaborating on documents.
Microsoft Teams is designed for Microsoft’s enterprise customers, and it includes enterprise-level security with two-factor authentication, single sign on through Active Directory, and data encryption. Teams is available for Windows, Mac, Android, iOS, and the web.
Ahead of Microsoft’s announcement, competing chat platform Slack took out a full page ad in the New York Times welcoming Microsoft to the chat space, offering some “friendly advice,” and signaling that it’s worried about competition from Microsoft.
In the piece that ends with a warning that “Slack is here to stay,” Slack says an open platform, love, and thoughtfulness and craftsmanship are essential to a successful communication product.
One final point: Slack is here to stay. We are where work happens for millions of people around the world.
So welcome, Microsoft, to the revolution. We’re glad you’re going to be helping us define this new product category. We admire many of your achievements and know you’ll be a worthy competitor. We’re sure you’re going to come up with a couple of new ideas on your own too. And we’ll be right there, ready.
A preview of Microsoft Teams is available in 181 countries and 18 languages starting today for Office 365 enterprise customers (Business Essentials, Business Premium, El, E3, and E5). It will officially launch early next year.
Tags: Microsoft, Slack, Microsoft Teams
Discuss this article in our forums
Second and Third-Generation Apple TV Models Not Working for Some Users
Some second and third-generation Apple TV owners have started experiencing a mysterious bug over the last few days, which seems to have essentially disabled the device for a number of users. Affected Apple TV models are only able to display Computers, Music, and Settings, with no other channel options available.
Customers who are seeing the issue have had all of their menu options suddenly disappear, leaving them unable to watch Netflix, Hulu, and other Apple TV channels.
Image via MacRumors reader Gabbisonn
This seems to be a bug that’s been around for awhile, but has become more prevalent over the last day or two. MacRumors has received several complaints via email, and there are threads outlining the problem both on our forums and on Reddit.
Restarting the Apple TV, resetting a router, and resetting the Apple TV don’t seem to work. A region change offers a temporary solution, but ultimately, the channels disappear again for many users. Some users have partially fixed the issue by changing DNS settings, but this doesn’t work for all users. A MacRumors reader describes the problem:
All I see is a computer and settings option. In the settings option I have access to everything except the main menu setting. There’s absolutely nothing in there as an option. I’ve restarted my Apple TV as well as my AirPort Extreme. I’ve reset the Apple TV and I inserted all the info back in like my iCloud acct etc. it obviously is logging onto the Internet since I was able to connect my iCloud account. One other strange thing is that my iTunes won’t recognize the Apple TV when I plug it in directly to my Mac mini via a USB cable. It just doesn’t show up. What the heck happened? Apple do you have an explanation for this one?
Not all Apple TV users are affected by the issue, but it appears to be fairly widespread. Only the Apple TV 2 and 3 are impacted — the fourth-generation Apple TV is functioning fine.
Apple’s support staff told a Reddit user that a fix was in the works, and to expect a software update in the near future. A MacRumors reader received similar information, with Apple suggesting a fix would be pushed within the next few hours.
Related Roundups: Apple TV, tvOS 10
Buyer’s Guide: Apple TV (Caution)
Discuss this article in our forums
Focus your mind with a lifetime subscription to Brain.fm’s audio library, now 80% off
Life is busy and relaxing can be hard. Avoiding distractions isn’t easy, they are literally everywhere. Whether you need to avoid them to get some studying done or need help getting yourself to sleep, it can be a struggle to get it done. There are a number of different programs and options out there to tune out these distractions, but unfortunately they aren’t all cheap.
Luckily, Brain.fm is a great way to tune them out at an affordable cost. For just $39 you can grab a lifetime subscription that will help you boost your productivity and get that to-do list accomplished.
With this lifetime subscription you’ll be able to:
- Choose whether you’re trying to work, relax, or sleep, & experience an original composition specially generated for that scenario
- Adjust the stream to play for 30 minutes, 1 hour, 2 hours, or indefinitely until you turn it off
- Explore different recordings & audio tracks for each category on your own
- Access premium-only content & track your work progress
This huge 80% savings that brings the price to just $39 won’t last too long, so you’ll want to act quickly. Whether you need to buckle down and study for that upcoming exam or just need to relax after a long day of work, Brain.fm is the perfect option for you.
Don’t wait for the price to jump back up to $200 and instead grab this lifetime subscription for yourself right now.
See at Android Central Digital Offers
Android security chief: There’s no doubt Google Pixel is as secure as the iPhone
We’ve been saying it for years, but Android’s director of security, Adrian Ludwig, put it on the record.
At the O’Reilly Security Conference in New York City, Ludwig told Motherboard that there’s no doubt that the Google Pixel and the iPhone are on par when it comes to security. “For almost all threat models…they are nearly identical in terms of their platform-level capabilities.”
He added, “In the long term, the open ecosystem of Android is going to put it in a much better place.”
Ludwig also mentioned that although Android’s security may have improved overall in the last year, the onus remains on carriers and manufacturers to deploy timely software updates and security patches. “We got quite a bit of work left to do to get to a point where that actually happens on a regular basis across the whole the ecosystem.”
OK Google, where’s your Amazon Echo Dot competitor?
For some reason, Google is still playing catch up to Amazon.
While undoubtedly immature and arguably not ready for prime time just yet, Google Assistant is easily the best of the virtual assistant platforms you can get today. The natural language processing is amazing, handling accents is beyond compare, and when Google Home launches this will be the first assistant that actually works as a whole life platform. Being great on the go and being good enough for a beta in the home is an impressive step for Google, but it’s easy to see how Amazon’s shadow looms over the Google Home efforts for the foreseeable future.
More: Google Home vs. Amazon Echo: The battle to control your home
Step one in solving this problem isn’t a more complete software package, though that absolutely also needs to happen. Instead, Google needs to respond immediately to the thing that makes Amazon more compelling than Google Home — price. Specifically, the price of Amazon’s second offering in the Echo space, the Dot.
Echo Dot isn’t just a smaller, cheaper Echo; in many ways, it’s a compelling argument for being the only Amazon Echo you should buy. Many early adopters have the bit Amazon Echo in a centrally located place in the house, somewhere that everyone can call to and get a good response when needed. The impressive microphone and decent speaker in Echo makes this fairly easy to do, and so when Echo Dot was released those users started thinking about the kitchen, the bedroom, and anywhere else the first Echo didn’t quite reach. It’s a solid plan, but hardly one that needs to be followed by people who haven’t purchased the original Echo.
Basically, the creator of the $35 Chromecast Audio could easily step up and make a $50 or $60 Google Home Mini.
At $50, you can have three Echo Dots for the cost of one Echo. They’re cheap enough that Amazon will give you one for free if you buy six, and people are actually taking the company up on that deal. That’s seven Echo systems with the same impressive microphone system that are easier to hide in more discrete places around your home, and can be connected to speakers you already have. Not that you necessarily need to, mind.
See Dot at Amazon
If you’re not listening to music on Echo the speaker built in to Dot is more than enough to acknowledge commands or listen to flash briefings. An Echo isn’t required to power an Echo Dot, either, so it just makes sense in many homes to buy a couple of these smaller systems instead.
Echo Dot isn’t just a smaller, cheaper Echo. It actually makes a compelling argument for being the only Amazon Echo you should buy.
And that brings us back to Google Home, the $130 Google Assistant box that looks much nicer than Echo when positioned strategically, and is sold in multiples to make it easier to position around the house, but is still more than twice the cost of the Dot. It’s not hard to see why, after using one, Google worked hard to make sure the speaker not only sounds better than the big Echo but also works with other Home units to produce music throughout the house. That’s something Echo doesn’t do at all right now, and when combined with Google’s streaming partners has the potential to be a huge feature for music fans that don’t have a household stereo system already.
Basically, the creator of the $35 Chromecast Audio could easily step up and make a $50 or $60 Google Home Mini that speaks to me when I called. Something that doesn’t require me to walk around the house with my phone at all times, that my kids could use when I’m not around, and that was actually available in every room at an affordable price.
Unlike Amazon’s larger Echo, there’d still be a big reason to purchase one or more Google Home over this Mini strategy thanks to Google Cast, but it’d be a huge step toward offering a complete response to Amazon Echo instead of feeling like the biggest search company in the world was always weirdly one step behind taking over my home.
Troubleshooting your Samsung Gear VR starts with our Ultimate Guide!
Everything you need to keep your Gear VR running well!
There’s a lot of great things you can do inside of a Samsung Gear VR, but like any virtual reality system one small thing can take a great deal away from the experience. It’s not always easy to find a quick solution to one of those problems, so we’ve compiled a complete list of all the things you need to know if something happens. Troubleshooting your Gear VR is easy if you’ve got a guide handy,and that’s where we come in.
Here’s the ultimate guide to troubleshooting your Gear VR!
Read more at VR Heads!
L’Oreal turns to virtual reality to train hairdressers
Out-of-town trainings and classes can cost aspiring hairdressers a pretty penny. L’Oréal’s high-tech alternative could change that, though, and could even make learning more fun. The French cosmetics company teamed up with VR software maker 8i to create a new virtual reality curriculum for the Matrix Academy, L’Oréal’s training program for fledgling hairstylists. Together, the two developed an immersive, room-scale VR experience where trainees can walk around and observe a virtual hairdresser style a virtual client’s hair from every angle. They can even step into the hairdresser’s position to get a first-person view of the process. That sounds so much better than watching YouTube tutorials, doesn’t it?
Prior to teaming up with L’Oréal, 8i was having issues creating virtual hair, especially the frizzy type. But the two managed to find a way to fix that. 8i uses off-the-shelf cameras and automated software to record real hairstylists and models in order to create holograms out of them. The results, according to VentureBeat, are 3D photorealistic recordings, which they’ve presented at the Fast Company Innovation Festival.
This represents one of the many possible ways to take advantage of VR’s capabilities outside of consumer-related applications. Businesses are starting to tap the power of VR — a British financial institution, for instance, recently decided to interview potential employees by having them accomplish tasks in virtual environments. L’Oréal’s VR-based curriculum will initially be available in 25 Matrix Academies in the US, but the cosmetics giant has plans to deploy it worldwide.
Source: VentureBeat, Fast Company
AIVAnet 