Getting digitally a-head
How close should a likeness be?
Still from film Final Fantasy: The Spirits Within 2001
Our heads, in particular our faces are a key feature in how we present ourselves to the world, how we are recognised and how we are known. Before photography, and it would require a skilled painter or photographer to ‘capture a likeness’ of someone, and after they died, that is how they would be known for the rest of history. Digital photography and video has made images of ourselves common place, but until recently our avatars (representations of ourselves who can move and speak) have been crude or cartoonish. Partially this is because of phenomenon called the “Uncanny Valley”, we are very comfortable with cartoonish representations of humans as we are with photographic ones. But when that likeness is close, but not quite a photograph it can evoke an eerie repulsion. An early example of this was the 2001 animated film, Final Fantasy: The Spirits With, which was an extraordinary technical achievement, but wasn’t well received because the almost realism of the animation made people uncomfortable. Now we have become quite comfortable with the quality of these representations of people in computer games, and with the much more realistic (although controversial) inclusion of digital humans with real ones in films.
The creation of rigged (able to be animated) 3D digital likeness has up until now been the highly skilled and technical domain of VFX Artists. But for about two years software systems have been available that make making a realistic digital doppelgänger of yourself much more accessible. For this project we used a tool called Metahuman https://metahuman.unrealengine.com, I’ve been working with computers a long time, I’m not a VFX artist, but decided to have a crack (how hard could it be?)
Metahuman Creator is accessible through a web browser and with the starter content you can create unique and realistic digital human likenesses, and if you have a head and face that doesn’t stray too far from the norm then you could use this. We don’t. Other approaches involve starting by creating a 3D model of the skull, but not by using 3D modelling software (it takes too long, and I don’t have the skills) or high end 3D scanners that big companies have access to.
Unreal has a free iPhone app called Live Link Face that works if you have an iPhone 11 or newer. The reason it’s for iPhone 11 or newer is that they have built in depth camera, whereas android phones don’t. The Live Link app can be used to ‘puppet’ a metahuman character, i.e. transfer the facial expressions of a human via the camera to the metahuman. It can also be used to create a 3D scan of a head that can be turned into a metahuman using Unreal Engine. Works great if you aren’t old with an odd shaped head.
So I tried some different techniques. The first was to use photogrammetry to create a model of my head. Photogrammetry is a technique whereby you take multiple photos of a head from different angles and then software finds matching points between the photos and triangulates into a point in space.
photogrammetry source images
The other technique is to use a manual form of photogrammetry, using just a few images, trace outlines of key features, like ears, mouth, eyes and nose, as well as the shape of the skull, and match those between images. We used Keentools FaceBuilder but it still required quite a bit of learning and experimentation to eventually make something like my odd shaped head.
Head Model built with Keentools Facebuilder - notice that the hair is flat, its part of the ‘skin’ texture
Once there is a good base model, it is used as a base to build the ‘rigged’ Metahuman model, again for odd shaped heads, it’s hard to get to fit. Below are some of the attempts.
One of my favourite Bill’s, but he’s a bit young and has a very red nose.