Qualcomm’s new depth-sensing camera is surprisingly effective

Dual cameras are so passé. Qualcomm is getting ready to define the next generation of cameras for the Android ecosystem. It’s adding three new camera modules to its Spectra Module Program, which lets device manufacturers select ready-made parts for their products. The additions are an iris-authentication front-facing option, an “Entry-Level Computer Vision” setup and a “Premium Computer Vision” kit. The latter two carry out passive and active depth-sensing respectively, using Qualcomm’s newly revamped image signal processing (ISP) architecture.

Of the three new modules, the most intriguing is the premium computer vision kit. That option is capable of active depth sensing, using an infrared illuminator, IR camera and a 16-megapixel (or 20-MP, depending on configuration) RGB camera. The illuminator fires a light that creates a dot pattern (using a filter), and the IR camera searches for and reads the pattern. By calculating how the dots warp over a subject and the distance between points, the system can tell how far away something is. And since this technology uses infrared light, it can also work in the dark.

During a recent briefing in New York, Qualcomm representatives also showed us a video (embedded above) of the setup in action. What was truly remarkable was how the system was able to accurately display the profile or a person’s hands playing the piano, despite only shooting from above. We’ll have to wait and see it in action for ourselves before knowing if it’ll be effective in the real world, but so far the technology is impressive.

The module can get very detailed, since it uses more than 10,000 points of depth and can discern up to 0.125mm between the dots. This precision is important. “Depth-sensing is going to be mission critical going forward,” Qualcomm’s product marketing lead for camera and computer vision Philip-James Jacobowitz told Engadget.

There are plenty of useful applications for depth-sensing — one of the most widespread being creating artificial depth of field in images. It can also help in face detection, recognition and authentication, 3D object reconstruction and localization and mapping, according to Qualcomm.

If this is all starting to sound very familiar, you might be thinking of Intel’s RealSense. It’s used more commonly in laptops for facial or iris authentication via Windows Hello, although it has made its way to a few phones (and even drones) recently. But Qualcomm said that according to its understanding, the thermal overhead on Intel RealSense is a barrier to entry for smartphones. That means it most likely runs too hot to work well in a handset. Regardless of the differences in equipment, it’s clear depth-sensing has a wide variety of useful applications, and Qualcomm working on a competing option could help improve performance across the board.

That’s not to say the two other modules Qualcomm announced today aren’t useful. The entry-level computer vision option, for instance, allows for similar (albeit less-precise) features at a lower price. But it consumes more power than the premium kit. And according to Qualcomm, its iris authentication module can read your eyes even when you have sunglasses on — something the company’s representatives demonstrated effectively at the briefing.

No word yet on exactly when we can expect the new cameras to arrive in devices we can play with, but Qualcomm says the new modules and ISPs “are expected to be part of the next flagship Snapdragon Mobile Platform.” Smartphone makers have to incorporate the components into their products first, too, meaning we’ll probably have to wait for next year’s phones.