"It's not just about cutting-edge graphics or advanced motion tracking. It's about making it as natural and effortless to interact with people in virtual reality as it is with someone right in front of you. The challenge lies in creating authentic interactions in artificial environments"
Yaser Sheikh, Facebook Reality Labs Director
Yaser Sheikh, Facebook Reality Labs Director
Creating a perfect avatar boils down to achieving what the team calls 'social presence', and vaulting the uncanny valley to deliver acceptably realistic avatars is something they've been working on for years. The team calls the process "passing the ego test and the mother test."
"You have to love your avatar and your mother has to love your avatar before the two of you feel comfortable interacting like you would in real life. That's a really high bar," Sheikh maintains.
A demonstration showing two VR users talking with lifelike avatars gives an interesting look at what the future of VR avatars could be.
The company says at this point these sorts of real-time, photorealistic avatars require quite the gear to achieve. The lab's two capture studios—one for the face, and one for the body—are admittedly both "large and impractical" at this point.
"You have to love your avatar and your mother has to love your avatar before the two of you feel comfortable interacting like you would in real life. That's a really high bar," Sheikh maintains.
A demonstration showing two VR users talking with lifelike avatars gives an interesting look at what the future of VR avatars could be.
The company says at this point these sorts of real-time, photorealistic avatars require quite the gear to achieve. The lab's two capture studios—one for the face, and one for the body—are admittedly both "large and impractical" at this point.
The ultimate goal however is to achieve all of this through lightweight headsets, although Facebook Reality Lab currently uses its own prototype Head Mounted Capture systems (HMCs) equipped with cameras, accelerometers, gyroscopes, magnetometers, infrared lighting, and microphones to capture the full range of human expression.
Using a small group of participants, the lab captures 1GB of data per second in effort to create a database of physical traits. In the future, the hope is consumers will be able to create their own avatars without a capture studio and without much data either.
At the moment volumetric captures last around 15 minutes, and require a large number of cameras to create the most photorealistic avatars possible. The lab then plans to use these captures to train AI systems so consumers could then quickly and easily build a Codec Avatar from just a few snaps or videos.
There are still plenty of challenges to address on the way there. One big problem looming on the horizon is 'deepfakes', or the act of recreating a person's appearance or voice to deceive others.
Using a small group of participants, the lab captures 1GB of data per second in effort to create a database of physical traits. In the future, the hope is consumers will be able to create their own avatars without a capture studio and without much data either.
At the moment volumetric captures last around 15 minutes, and require a large number of cameras to create the most photorealistic avatars possible. The lab then plans to use these captures to train AI systems so consumers could then quickly and easily build a Codec Avatar from just a few snaps or videos.
There are still plenty of challenges to address on the way there. One big problem looming on the horizon is 'deepfakes', or the act of recreating a person's appearance or voice to deceive others.
"Deepfakes are an existential threat to our telepresence project because trust is so intrinsically related to communication. If you hear your mother's voice on a call, you don't have an iota of doubt that what she said is what you heard. You have this trust despite the fact that her voice is sensed by a noisy microphone, compressed, transmitted over many miles, reconstructed on the far side, and played by an imperfect speaker." Yaser Sheikh, Facebook Reality Labs Director
Sheikh maintains we're still years away from seeing this level of avatar photorealism, although the lab is currently exploring the idea of securing future avatars through an authentic account, as well as several security and identity verification options for future devices.
Sheikh maintains we're still years away from seeing this level of avatar photorealism, although the lab is currently exploring the idea of securing future avatars through an authentic account, as well as several security and identity verification options for future devices.
IF YOU WANT TO LEARN MORE: