Problems, problems, problems (Part4)

3 weeks to go till hand in day. We mainly did tests in the first couple of weeks which was a good move in retrospective considering the amount of problems we had (hence the title of this entry). It should go smoother from now on.
One the most difficult shots we had planned involved 3d tracking. Since Maya 2010 comes with MatchMover we decided to use it for 3D motion tracking of a green box which will later be replaced with a CG gun. Oddne built a huge box out of wood and painted it green with car paint. To see his progress check out his blog http://oddne.blogspot.com/.
The box is quite heavy and like I said quite big. To handle it is difficult and that is a good thing since it will be replaced with a huge bulky metal gun. When it comes to shooting the scene we don't have to worry about acting like the gun is heavy.
We put massive crosses with black duck tape on all surfaces of the box for tracking markers and shot some test footage.

First of all you can see that we didn't use a tripod and a hand-held camera shot is really not good for this procedure. It's not impossible but you would have to track the environment as well which is especially in this case unnecessary work. I realised pretty soon after putting the footage into MatchMover that the crosses are far to big. MatchMover only needs contrast between pixels. The bigger the markers the more likely it is that the track jumps within the the massive amount of black, it just won't see a pattern any more but a big black surface. Then I realised that the tape is quite reflective so whenever it turns to where the light hits it it becomes white. That obviously isn't good for the tracking software which searches for colour differences between pixels so the tracking makers really shouldn't change colours. The main problem though is that I haven't used a 3D tracker before neither have my team mates nor any of my class mates for that matter. Although Georg has used 3D tracking software before he never used MatchMover. So I went home and did all the tutorials I could find and read the manual. Unfortunately there isn't much about 3d motion tracking in the manual or online. 3D tracking software is mainly used to imitate a camera track. The manual had a section about tracking moving objects so I followed that route. It says I would have to make a mask over the moving objects (in this case me and the box) then auto track the environment, invert the mask to only track the moving object(in this case only the box) and track this information into a different group. After a couple of tries I managed to solve the 3D camera, but the points where far from accurate. Frustrated and tired I returned to uni on the next day only to find out that Georg found an easy way to track objects by importing a mesh into the scene and attach its vertices to a tracking point. I tried it and it didn't work but only because I couldn't track enough points to solve a camera. All in all I would say that this test footage is absolutely useless. But I guess that's what test footage is for. I did pretty much every mistakes you could do when preparing a shot for 3D tracking, but at least I know better now.
So when the day of the shooting came I measured the box, built the exact replica in Maya with a few more divisions on each face evenly spread and measured the distance between each vertices to tape tiny squares of insulation tape(smaller and less reflective than duck tape!)to the right place onto the box.

That is the shot we're going to use. I'm going to talk about the experience of the green screen shoot in another entry. The footage looked alright but I still wasn't looking forward tracking it. I haven't exactly had a successful solve yet and the amount of hours that go with it making the whole tracking experience a rather unpleasant matter.
I locked myself in my room (not literally) for the night with a few energy drinks and a big pouch of tobacco. 5 hours later, all the energy drinks gone and my tiny room stinking of fags, I had 45 hand tracked points. While waiting for the solve to finish I tried to think of ways to still make the shot if it doesn't work (which I was expecting). Once it finished I changed to the 3D view to check the mess I just made and I saw this...

IT WORKS!

Ok there is a little glitch when he picks it up, but that can be easily be fixed in Maya.

It was 3am by then and I had to get up at 7 to go to but it was so worth it.

I had to wait until after the weekend to show Oddne, Simon and Georg my progress. We obviously then had to try how it looks with the CG gun on top of it so I exported the the MatchMover scene into Maya and parented the gun to the locators and after a bit of tweaking I got that!

Here is the MatchMover scene in Maya. The 3D tracking points have been exported as locators.

And this is a rough version of the gun parented to those locators.

I think it looks quite good already. I will have to tweak the position of the gun a bit and get rid of those twitches by baking the keyframes and then manually correct them, but I think we're onto something good here.

Unfortunately there is a part 5 of my "Problems, problems, problems" blog series. This time it's the empty bullet shelves flying out of the gun using particles and instancers.