Enlarge / Instead of
transmitting an image for every frame, Maxine sends keypoint data
that allows the receiving computer to re-create the face using a
neural network. (credit: Nvidia)
Last month, Nvidia
announced a new platform called Maxine that uses AI to enhance
the performance and functionality of video conferencing software.
The software uses a neural network to create a compact
representation of a person’s face. This compact representation can
then be sent across the network, where a second neural network
reconstructs the original image—possibly with helpful
Nvidia says that its technique can reduce the bandwidth needs of
video conferencing software by a factor of 10 compared to
conventional compression techniques. It can also change how a
person’s face is displayed. For example, if someone appears to be
facing off-center due to the position of her camera, the software
can rotate her face to look straight instead. Software can also
replace someone’s real face with an animated avatar.
Maxine is a software development kit, not a consumer product.
Nvidia is hoping third-party software developers will use Maxine to
improve their own video conferencing software. And the software
comes with an important limitation: the device receiving a video
stream needs an Nvidia GPU with tensor core technology. To support
devices without an appropriate graphics card, Nvidia recommends
that video frames be generated in the cloud—an approach that may
or may not work well in practice.
Source: FS – All – Science – News
Nvidia developed a radically different way to compress video