Now to the testing!
Orla did a quick stop motion test. Because of technical difficulties, we had to shoot it in low res (788 x 576), while the life action footage is shot in HD 1080.
Here's the stop motion test!
Here's the life action front shot!
So... I tracked the stop motion test, but unfortunately I (again) messed up the set up. There were not enough tracking points to track all of the movement, without any jerky glitches. The first 2, 5 seconds are pretty much useless. Well I guess that's why we do the testing. The last 6 seconds were fine though, so at least I had something to play with.
I prepared (stabilized and masked) each eye in After Effects and exported them as 32bit EXRs.
Then modeled the geometry according to the 3D location of the locators exported from the 3D motion track. Because MatchMover allows you to animate the camera rather than the scene, it's possible that the geometry can just sit in 3D space, while the camera moves around it, making it look like the geometry is moving.
This is the geometry for the projection (not perfect. Gotta spend more time on it once it comes to production).
Next thing I build the set up for the camera projection and painted transparency onto the mesh using Maya 3D paint tool! Here's a screen shot of the geometry with the life action footage projected onto it!
The main thing here is that the camera is already tilted, but because the 2d(!) footage is projected onto 3D geometry it still looks 3D.The shading and lighting is going to be a bit iffy. But I tried to simulate the lighting.
Maya's viewport allows to see the camera projected geometry displayed and updated, which makes it easier to set up the mesh and the cameras.
That's all the work done in Maya. Now it's comping!
I rendered out each eye and one render of the mesh with a lambert material override to get some shading. The comping mainly involved color grading, masking out the nose and adding some grain to the CG shots, since the resolution was much higher (although I think I got a bit carried away with that).
I also had to paint out some bits of the puppets eyebrows, that woulg stick out at the beginning when it turns around.
There're still some things to sort out, but the main idea stands. The test was mainly to go through the pipeline and to proof to Sarah that it could be done. She's not that familiar with 3D so she didn't trust the idea at start. But I think I managed to convince her that this technique is feasible and will save her a lot of time over the masking technique from Madame Tutli-Putli.