Over the past two years, Facebook’s artificial intelligence research team (also known as FAIR) has been hard at work figuring out how to make computer vision as good as human vision. The crew has made a lot of progress so far (Facebook has already incorporated some of that tech for the benefit of its blind users), but there’s still room for improvement. In a post published today, Facebook details not only its latest computer-vision findings but also announces that it’s open-sourcing them to the public so that everyone can pitch in to develop the tech. And as FAIR tells us, improved computer vision will not only make image recognition easier but could also lead to applications in augmented reality.
There are essentially three sets of code that Facebook is putting on GitHub today. They’re called DeepMask, SharpMask and MultiPathNet: DeepMask figures out if there’s an object in the image, SharpMask delineates those objects and MultiPathNet attempts to identify what they are. Combined, they make up a visual-recognition system that Facebook says is able to understand images at the pixel level, a surprisingly complex task for machines.
“There’s a view that a lot of computer vision has progressed and a lot of things are solved,” says Piotr Dollar, a research scientist at Facebook. “The reality is we’re just starting to scratch the surface.” For example, he says, computer vision can currently tell you if an image has a dog or a person. But a photo is more than just the objects that are in it. Is the person tall or short? Is it a man or a woman? Is the person happy or sad? What is the person doing with the dog? These are questions that machines have a lot of difficulty answering.
In the blog post, he describes a photo of a man next to an old-fashioned camera. He’s standing in a grassy field with buildings in the background. But a machine sees none of this; to a machine, it’s just a bunch of pixels. It’s up to computer-vision technology like the one developed at FAIR to segment each object out. Considering that real-world objects come in so many shapes and sizes as well as the fact that photos are subject to varying backgrounds and lighting conditions, it’s easy to see why visual recognition is so complex.
The answer, Dollar writes, lies in deep convolutional neural networks that are “trained rather than designed.” The networks essentially learn from millions of annotated examples over time to identify the objects. “The first stage would be to look at different parts of the image that could be interesting,” he says. “The second step is to then say, ‘OK, that’s a sheep,’ or ‘that’s a dog.’
“Our whole goal is to get at all the pixels, to get at all the information in the image,” he says. “It’s still sort of a first step in the grand scheme of computer vision and having a visual recognition system that’s on par with the human visual system. We’re starting to move in that direction.”
By open-sourcing the project on GitHub, he hopes that the community will start working together to solve any problems with the algorithm. It’s a step that Facebook has taken before with other AI projects, like fasText (AI language processing) and Big Sur (the hardware that runs its AI programs). “As a company, we care more about using AI than owning AI,” says Larry Zitnick, a research manager at FAIR. “The faster AI moves forward, the better it is for Facebook.”
One of the reasons Facebook is so excited about computer vision is that visual content has exploded on the site in the past few years. Photos and videos practically rule News Feed. In a statement, Facebook said that computer vision could be used for anything from searching for images with just a few keywords (think Google Photos) to helping those with vision loss understand what’s in a photo.
There are also some interesting augmented reality possibilities. Computer vision could identify how many calories are in a photo of a sandwich, for example, or it could see if a runner has the proper form. Now imagine if this kind of information was accessible on Facebook. It could bring a whole new level of interaction to the photos and videos you already have. Ads could let you arrange furniture in a room or try on virtual clothes. “It’s critical to understand not just what’s in the image, but where it is,” says Zitnick about what it would take for augmented reality applications to take off.
Dollar brought up Pokémon Go as an example. Right now the cartoon monsters are mostly just floating in the middle of the capture scene. “Imagine if the creature can interact with the environment,” he says. “If it could hide behind objects, or jump on top of them.”
The next step would be to bring this computer-vision research into the realm of video, which is especially challenging because the objects are always moving. FAIR says that some progress has already been made: It’s able to figure out certain items in a video, like cats or food. If this identification could happen in real time, then it could theoretically be that much easier to surface the Live videos that are the most relevant to your interests.
Still, with so many possibilities, Zitnick says FAIR’s focus right now is on the underlying tech. “The fundamental goal here is to create the technologies that enable these different potential applications,” he says. Making the code open-source is a start.
The first virtual reality film to feature President Obama is, not surprisingly, a love letter to some of America’s greatest treasures: its National Parks. Together with Oculus, National Geographic and the VR studio Felix & Paul, the President filmed Through the Ages, a VR experience meant to celebrate the centennial of the National Park Service.
The film shows off some grand sights from Yosemite, including the El Capitan mountain range and Mariposa Grove’s sequoia trees. And we also get a chance to see President Obama exploring the sights with his family. Most importantly, it issues a strong environmental message focused on preserving the parks for future generations.
Through the Ages is available today free on the Oculus Store for Gear VR and Oculus Rift owners, as well as a 360-degree video on Facebook. As you’d expect, the film takes full advantage of VR’s immersiveness — or at least, as much as it can being a mere video recording, rather than a fully rendered environment. It was shot stereoscopically, so there’s a greater sense of depth to the images compared to most other 360-degree videos, making you feel as if you’re actually standing atop mountains or right below the tallest trees in the world. It’s also one of the sharpest virtual videos I’ve ever seen, with none of the mudiness we saw in the Olympics’ VR videos.
Felix Lajeunesse and Paul Raphael, the founders of the VR studio producing the film, wouldn’t give up any specifics on how they shot the film, but they revealed their rig is a “custom designed body with multiple sensors” and a variety of lenses. At this point, their equipment can deliver the equivalent of “3.5K” video resolution to each eye, but of course the actual video quality is limited to the VR equipment you’re using (most recent Samsung phones use displays with 2,560 by 1,440 resolutions, while the Oculus Rift delivers 2,160 by 1,200 across both of its lenses). Still, Felix & Paul will be able to offer higher quality versions of the film as newer VR tech hits the market.
Ultimately, the film gives people who might not get a chance to visit Yosemite a way to experience brief moments of its grandeur. President Obama mentions at one point that visiting the park as a child was a life-changing experience, and his hope to preserve it feels genuine. Sure, a VR film won’t replace actually trekking to Yosemite in person, but it’s the best option you’ll have until making that trip.
By coordinating more with Facebook, we’ll be able to do things like track basic metrics about how often people use our services and better fight spam on WhatsApp. And by connecting your phone number with Facebook’s systems, Facebook can offer better friend suggestions and show you more relevant ads if you have an account with them. For example, you might see an ad from a company you already work with, rather than one from someone you’ve never heard of.
Existing WhatsApp users can choose not to share their account information with Facebook. On the iPhone app, before you tap “Agree” to accept the updated terms, tap on “Read,” scroll to the bottom, and toggle the control. Users that agree to the updated terms also have an additional 30 days to opt out by going to Settings > Account > Share My Account Info and toggling the appropriate control in the app.
WhatsApp is free on the App Store [Direct Link] for iPhone.
Tags: Facebook, WhatsApp
Discuss this article in our forums
If you want to post a video on the web several years ago, YouTube was the go-to spot. Now, Google’s video network is feeling the pinch with strong video features from Facebook, Twitter and others, and has decided to push back, according to Venture Beat. The feature, internally code-named “Backstage,” will reportedly allow users to share photos, short posts, links, polls and videos with subscribers. Much like a Facebook timeline, items will be listed from newest to oldest and posted in subscribers’ feeds.
Backstage, which will appear as a channel tab, gives producers a new way to share content with fans. But it will also allow subscribers to comment with (Backstage-only) video, photos and other “rich replies,” according to VB. That’s along the same lines as Twitter, which supports links, videos and GIFs. It could also open it up to more abuse, though it’s hard to top a YouTube comments section for that.
YouTube is still by far the most popular video site on the net. Facebook recently reported that users watch 100 million hours of video per day, but YouTube reportedly serves up over 500 million hours daily. There’s often not much reason to linger on YouTube (other than watching more videos), though, so the site is likely hoping the social aspect will convince viewers to stick around longer. Backstage is expected to arrive by the end of the year, starting with select, influential YouTube accounts.
Source: Venture Beat
WhatsApp announced a major change that we suspected was coming today by adding terms that allow it to share user data with its parent company Facebook.
Back in January, code showed up suggesting a closer sharing of data between the two companies, and now it’s arrived whether you wanted it or not. Privacy advocates will clearly be concerned about the sharing of data between two of the worlds’ most popular social services, and the two largest messengers.
WhatsApp, on the other hand, says it needs to share data to test out new features in the next couple of months, like “ways for you to communicate with businesses that matter to you” and “hearing from your bank about a potentially fraudulent transaction, or getting notified by an airline about a delayed flight.”
It’s worth noting the details included in the agreement: your phone number, profile name and photo, online status and status message, last seen status, and receipts. That means read receipts and sent receipts, if the option is switched on. It doesn’t include the content of your messages.
If it all makes you a bit uneasy, try not to worry, it’s just the biggest social company in the world working out better ways to target its ads — and there is a way to opt out, but it could be easier.
Via: The New York Times
The decision to get a high-end virtual reality headset is as much about the software selection as the technology itself. So which platform is getting the most attention from developers? Apparently, it’s HTC’s Vive. A UBM Game Network industry report shows that 49 percent of VR developers are targeting the Vive, while 43 percent are writing software for the Oculus Rift. And the gap gets wider when it comes to the next game from these studios — nearly 35 percent are building for the Vive, while a little over 23 percent are aiming at the Rift.
The study doesn’t explain why the Vive is getting more support, although its technology may play an important role. While Oculus is largely focused on sit-down VR with conventional controls (its motion controller won’t arrive until later this year), the Vive shipped from the start with support for room-scale VR and motion input. There’s just more you can do. We’d add that the Vive already has unique experiences, like the Star Wars VR experiment, and that HTC has managed to get the Vive into the hands of influential YouTube stars like PewDiePie. If many of your potential players were most excited for the Vive, which one would you support? Still, it comes as a mild surprise when Oculus has the luxuries of both years of publicity and Facebook’s financial backing.
There are plenty of challenges for developers, regardless of the hardware. The steep price of high-end VR (you need a fast PC on top of the headset) and a lack of must-have titles play a part, but one of the most common problems is nausea. As we found out first hand, sickness can sour an otherwise great experience — people might not try VR again if their first experience makes them queasy. Also, just under half of all VR creators are funding their projects with personal funds, rather than leaning on outside help.
Thankfully, there’s a lot of optimism. Nearly 96 percent of surveyed developers believe there’s a sustainable audience for VR and augmented reality. While that’s not completely shocking for a group that’s already committed (you wouldn’t make a VR game if you didn’t think people would buy it), the data shows that creators believe there’s a real, long-term audience.
It’s no secret that Mark Zuckerberg wants Facebook to become a more video-centric social network, and the company has been conducting various tests to see how its users would react to new features. One of its latest mobile app experiments is auto-playing videos on the News Feed… with sound.
According to Mashable, some affected users see an icon on videos that you can tap to toggle sounds or or off. That sounds manageable, and we can imagine people liking the feature. However, other testers are reporting that sounds automatically start up when videos play on their News Feed, so long as their devices aren’t on silent mode. That one sounds like a huge PITA. It appears that the test is only live for a small percentage of people on mobile in Australia, however, and it doesn’t seem like it will be expanding elsewhere just yet.
You’re probably well-acquainted with how videos work on the News Feed by now. They automatically (and silently) play while they’re visible on your screen, but they stop as soon as you scroll past them. The system’s pretty convenient for watching videos on the go, especially if you usually can’t be bothered to dig up your earphones. It will be tough browsing your friends’ posts in public places if sounds autoplay, as well.
If you’re Down Under and among the small number of users affected, you don’t have to deal with it if you don’t want to. You can always switch sounds off in Settings or mute your phone completely.
Here’s the statement we got from Facebook:
“We’re running a small test in News Feed where people can choose whether they want to watch videos with sound on from the start. For people in this test who do not want sound to play, they can switch it off in Settings or directly on the video itself. This is one of several tests we’re running as we work to improve the video experience for people on Facebook.”
Facebook isn’t done launching products designed to capture the Snapchat generation. Its latest attempt after Instagram Stories and live filters? A new standalone, video-centric social app for high school students called Lifestage. To be able to complete your profile, you’d have to take videos and selfies of your likes, dislikes and facial expressions. It will ask you take videos of your BFFs, to bust out dances moves on cam, take photos of your desserts, so on and so forth. When we say that it’s for high school students, we mean you won’t even be able to see other people’s profiles if you’re older than 22. That’s assuming you won’t creepily pretend to be younger than you are.
See, it only shows you profiles of other kids going to your school and other ones nearby, similar to how Facebook was in the beginning. Further, the app will only unlock profiles from your school if over 20 students sign up. While we’ll have to wait and see if the new social network catches on, Lifestage was created by someone who truly knows its audience: 19-year-old Facebook employee Michael Sayman, who’s been with Facebook since he got out of high school. He’s been making apps since he was 13 years old, and Mark Zuckerberg personally invited him to join his team.
Sayman says his app “looks back at the days of Facebook from 2004 and explores what can be done if we went back and turned the crank all the way forward to 2016 with video-first.” That certainly aligns with Zuckerberg’s plan to transition his website into a more video-centric network. There’s no word yet on when it’ll come out for Android devices, but iPhone- and iPad-using high schoolers can now download it from iTunes.
Source: Michael Sayman (Facebook)
Facebook has released a new teens-only social app called “Lifestage” that asks users to create profiles by uploading video clips instead of filling in text fields.
The standalone app is aimed at high school kids aged 21 and under, and doesn’t require a Facebook account. Users are asked to select their high school and are then shown video profiles of people at the same school or ones nearby, as long as at least 20 people from the same school use the app.
User profiles ask kids to upload videos of their “happy face”, “sad face”, likes, dislikes, best friend, the way they dance, and more, and Lifestage turns the clips into a video profile that others can then watch on the app’s social feed.
While there’s no restriction on who can download the app and create an account, anyone 22 or older will only be able to see their own profile, although Lifestage notes during sign-up that it can’t verify that users are the age they say they are. The app includes various highly visible blocking and reporting options, apparently to guard against the possibility of suspect users.
TechCrunch reports that the app was designed by Michael Sayman, a 19-year-old Facebook product manager who aims to replicate Facebook’s original incarnation as a college student network. “I wanted to work on an app that my demographic would relate to, or at least that my friends would want to use,” said Sayman.
The launch of Lifestage is certainly consistent with Mark Zuckerberg’s stated goal of putting video at the heart of all of Facebook’s apps and services, but it also points to the company’s continuing concern at Snapchat’s surging popularity among younger users, which has already led Facebook to imitate several of the app’s features in its photo-focused social offering, Instagram.
Time will tell whether Lifestage succeeds in attracting a younger crowd, or goes the same way as Poke, Slingshot, Paper, and Notify, all of which Facebook eventually binned following a lack of uptake.
Lifestage is a free download for iPhone and iPad on the App Store. [Direct Link]
Discuss this article in our forums
Bigscreen’s promise to bring the environment of a LAN party into virtual reality is becoming more credible, now that it’s also available in the Oculus store. The free software has been “completely cross-platform” since launch, ready for sharing with friends using Oculus Rift and HTC Vive VR headsets, and now you can get it in a new place. As the name implies, it syncs a virtual space so people can show what’s on their desktop to everyone else, even if they’re not physically looking over your shoulder.
The software is also getting a big upgrade, with avatars that are customizable down to their hair, eyes, skin color, gender and glasses. It also claims pseudo eye-tracking and synced mouth movements to help increase the presence in VR (at least, as much as you can with a group of disembodied heads). Another new feature is that now everyone will be able to share their desktop audio, not just the host. That makes it easy for anyone to pull up a YouTube clip, start playing a game or listen to music, without shuffling things around so everyone can hear.
The new update should be available on Steam and Oculus right now.
Source: Oculus, Steam