Marker removal

What's about? The purpose of this shot was to remove all the marker using the object tracking and build a 3D reconstruction to project some 2D element on the face.

What did I do? Tha main challenge was to remove the markers considering the light change on the face. For this reason it was not possible to use some static patch to cover the markers. It was necessary to stabilize the face movements in order to paint out the markers on a motion face instead. To do so, I used Nuke camera tracker to track the face movements. With the help of the point cloud I made a 3D reconstruction of the face modeling a 2D card in the 3D nuke environment. After that, I could stabilize the movements by unwrapping the UV and paint over to remove the markers. The card was necessary because the poisson mesh node in Nuke doesn't provide UV map.
In this way I had a geometry where I could project the scar and the bruises on. The first was made with a 2D image combined with a displacement on the geometry to have tiny parallax and the second with a procedural noise generation and grading.
In addition I also used the camera to export a 3D eye from maya and composite it as a glass eye.

What techniques did I use?

• Object tracking
• paint and reconstruction
• 3D projection
• CG integration
• 2D elements