Relightable Dome creates interactive images where you control the light source

Stitching together a bunch of images to create a single picture is hardly a new concept. Panoramas are old hat, and Google is even using 360-degree photos to help guide your shopping decisions. But a team of students at NYU’s Interactive Telecommunications Program, with the help of a few others, are taking a slightly different approach. The Relightable Dome, or Relightable Photobooth, uses Reflectance Transformation Imaging (RTI) technology which captures an object or person under a variety of lighting conditions then combine it into a single, interactive photo. Inside the box is a concave surface studded with 81 LED flashes. A custom microcontroller inside triggers each flash in sequence and a camera mounted at the rear captures an image at each stop. And, while the enclosure is custom, the camera doesn’t need to be. At the back is simply a bracket that practically any shooter can be mounted on, though in this case it was a Canon DSLR. The pictures are then stitched together on a laptop using a piece of open source software and a special algorithm to create an interactive image where you control the light source. The whole process, from triggering the first flash to final file takes less than five minutes.

The effect is quite different than a 3D model where you manipulate the object. In fact, we can only describe it as dramatic. Museums were among the first to make use of the tech for capturing detailed imagery of stone inscriptions, textiles and manuscripts. Shifting the light source exposes subtle nuances in the texture and illuminates even the deepest crevices, without creating a sterile flat image. While the uses for retailers and museums are obvious, something even more stunning happens when you stick your face inside. By simply shifting the angle of the light source you can bring emotion to an otherwise expressionless face. A flash from a high corner may cast menacing shadows and create the illusion of a frown or scowl, while one closer to face and slightly beneath the mouth may make it appear as if you’re smiling.

The project was partially inspired by Yang Jiang’s background in photography, and her obsession with lighting and the way shadows can play with the perception of the human face. Along with Peiqi Su, Liu Jing and Liao Wei, she worked to create this project for ITP’s Winter show, where it drew a lot of attention — almost every time we walked by someone was crouched with their head inside. The next step is potential commercialization. Right now, the only way to view the files is either with a desktop RTI viewer, or through the web view on the Relightable website. Eventually the group plans to build an embedable widget for viewing the images, but for now you’ll have to go to their site and upload images of your favorite Engadget editors directly.

Edgar Alvarez contributed to this report.

Filed under: Cameras

Comments

The Desolation of Smaug shows Peter Jackson still hasn’t perfected HFR

Peter Jackson’s introduction of high frame-rates (HFR) in Hobbit: An Unexpected Journey caused him an unexpected headache: many filmgoers hated the new tech. Those folks found that the hyper-realistic screenings ruined the magic, while others like myself enjoyed the eyestrain-free 3D experience and felt it should be given a chance, despite some issues. Because of all that, Jackson decided to tone down the too-sharp images for the HFR version of The Desolation of Smaug to make it more filmic and please critics and fans alike. With that in mind, I went to an HFR screening as an unabashed 3D buff, hoping to be absorbed by the story and not distracted by the tech. Unfortunately, it didn’t quite happen that way.

Why did nay-sayers dislike HFR on Hobbit: An Unexpected Journey? First off, many felt that the higher frame speed looked too real, revealing production flaws and ruining the suspension of disbelief. Worse, purists compared HFR to a cheap TV effect and called it vastly inferior to time-tested 24 fps. Though Jackson claimed to be unfazed by all the criticism, he did take it into account for Hobbit: The Desolation of Smaug. What audiences actually disliked about the first film, he felt, was not the HFR but the extra sharpness that resulted from filming digitally in 5K with Red’s Epic cameras instead of film. To remedy that, he told Variety that actor-friendly “Pro-Mist” lens filters were used during filming to apply a slight blur and remove any harshness during certain shots. In addition, each scene was carefully color corrected to “take the (video-like) HD quality away” and he deemed the final results to be “reasonably successful.”

To my eyes, those improvements did succeed in improving the oversaturated, oversharpened look of Hobbit: An Unexpected Journey. I was fine with action scenes in the first film, but often disappointed during slower dialogue parts thanks to sets, props and makeup that didn’t stand up to HFR’s unflinching gaze. Jackson’s tweaks mostly allayed those concerns for me and I didn’t notice any production design clunkers as I did in part one. Meanwhile, the 3D was magnificently integrated into the film and caused me no headache or other undue effects — which was the primary goal of the filmmakers with HFR.

So what’s the problem? The latest tinkering for Smaug can’t change the fact that a lot of cinematic sleight-of-hand gets tucked under the rug by the older film tech. With all its motion blur and flickering, 24 fps projection adeptly hides the styrofoam, plywood and even digital effects used to create film magic. If you crank up the reality quotient by spooling more frames, however, such fakery is easier to spot. Though Jackson succeeded in smoothing much of that over in the second installment, at times I still felt I was standing around a movie set rather than watching a film.

Part of this problem lies in the nature of 3D itself, which invites you to look around the frame and “step into Middle Earth,” as Jackson put it. During more ponderous scenes, I occasionally noticed sets or props that looked like, well, sets or props, that probably would’ve nicely passed for a sword or forest at regular speeds. In fact, Jackson and the studio elected to screen the film to critics and press at 24 fps so that they’d not get hung up on the technology — an apparent drop in confidence on his part. As a result, few reviewers ended up catching the 48 fps projections, but one who did felt (a bit harshly) that he was watching a behind-the-scenes video, and not the movie itself. Would other critics have had the same reaction had they seen the HFR screenings? I believe there would have been more than a few rants, and Jackson likely thought so too.

Perhaps because of these follies, Peter Jackson and pal James Cameron are left as the main proponents of HFR. Brian Singer, for one, thought about it for the next X-Men film but took a pass, telling THR he had “concerns about how certain sequences would look, and (the)… cost factor in rendering visual effects.” On top of money issues, doing CG at 48 fps can dramatically increase the time it takes to finish a movie — an ugly combo for filmmakers. If anybody was willing to take the plunge, extra profits are far from given, since many theaters screening Desolation of Smaug are charging the same amount for HFR as regular 3D viewings — with no proof that it draws extra viewers. The net result is that the cash and time outlay for high frame-rates isn’t worth the effort for filmmakers who don’t have their own digital effects workshop.

From what Peter Jackson’s said so far, the main reason for the existence of HFR is to make 3D better, which it undoubtedly does. Since he and James Cameron have played a large part in promoting 3D and getting theater owners onboard, it’s therefore easy to see why they’re also firmly behind HFR. They no doubt hope it’ll help boost 3D audience numbers, which have declined steeply since the salad days following Avatar. Given the waning audiences, bad press and steep technical hurdles, I believe Jackson’s last Hobbit HFR act will need to be as dramatic as the finale itself to save the fledgeling format.

Filed under: HD

Comments