https://www.youtube.com/watch?v=9D5ulvdg0jM
"Our system is capable of creating megapixel avatars from a single portrait
image."
Via Wayne Radinsky, who wrote:
"Megapixel portraits. I really like the way they take classical paintings and
animate them. The video shows you a diagram of how the system works, but it’s
still a bit hard to follow so I’m going to try to describe it in a different
way based on the paper.
Basically what they do is train a neural network on the “driver” video, where
it takes a frame as input and produces the same frame as output. This may sound
pointless but the neural network is challenged to build a 3D model of the
person to do this.
The “driver” here refers to the video that the “source” is going to be changed
to imitate. The “source” is the still image such as a classical painting that
the neural network will be producing a video in the style of. Once the neural
network is trained on the “driver”, it is then trained on the “source” even
though the “source” is a single frame.
The 3D model that is produced internally models two things, basically: head
rotations and facial expressions. There are a few more details necessary to be
part of the model but in the interest of being concise, I’m going to skip over
those. Basically what it needs to do to make the painting, or whatever the
“source” is, change in the manner of the “driver” video is to “3D warp” the
source image. So they created a 3D warping generator. This 3D warping generator
has to work using the head rotations and facial expressions data that come out
of the earlier analysis stage."
Share and enjoy,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
