Digital Visual eFfecX
Match Move, Assignment 3


The original input sequence, this image was taken from the online game "World of Warcraft"




Composting Result with the Horse-Gallop animation, and some static models, include the Chicken-Crossing model from SIGGRAPH

Team Member
R94725022 ¸êºÞ©Ò ¶À»²¤¤ 

Goal:
In this assignment, I try to blend a computer generated images with an existing video sequences. A necessary requirement is to estimate the camera motion parameters, and we want to reconstruct the video scene where the computer generated images can be seamlessly blended into the video as if it was really there.

Platform and Software
MS Visual Studio .Net 2003
gil2 graphic image library
Icarus
Blender 2.41
Python 2.4
Fraps
World of Warcraft
VirtualDub

Procedure:
1. Record a video
2. Use the Icarus to estimate camera parameters
3. Utilize Blender to render CGI using the parameters from Icarus
4. Blend the CGI video with the original one

Some Results
1. Input Video (somewhere near the IronForge)
2. Compositing result with static object, 1, 2
3. Compositing result with animation
4. Bad example for tracking (somewhere near Cenarion Hold, Silithus)
Details and Discussion:
1. Taking video REALLY is an important task. Instead of taking the video from real world, I shoot the video from a popular 3D game ¡§World of Warcraft¡¨, from Blizzard. The idea of recording an artificial world arises from the thought that in real world video we might get a lot of noise, where I can avoid that in 3D world on the other hand. In gaming industry, however, people use tricks to increase frame rate, such as LOD, and this can mislead the tracking process. Another problem with gaming environment is that we have to use MOUSE to move the camera, which is harder to control that motion than that of the hand-held camera. (Note: harder to control here means the camera motion is not natural, the hand-shake will more easily to cause the camera to move abruptly.)

2. Another problem with tracking arises where motion is large, and Icarus simply tells us that tracking failed. I guess they use KLT tracking, and the result of large-motion-tracking is terrible. As a consequence I made another video for slower motion, and the tracking improves dramatically.

3. For blending, the most challenging part I encountered is the compositing part. Tracking from Icarus provides rough motion, and we still see the CGI drifting in the result video. I have no idea whether this comes from the poor tracking result or I just not put the CGI attached to the ground properly. Anyway, it took a LONG, LONG time to tweak this part. Another part of blending is the shadow. I guess the result will improves significantly even we simply put the naive circular fake shadow.

4. The second time-consuming job in this assignment is the animation integration. Previously we were putting static object in the scene, and here I also made a video with an animation, the Horse-Gallop animation from CSAIL@MIT. Because Blender 2.41 did not provide an interface to load a sequence of animating model, I have to insert them FRAME-BY-FRAME, and put them properly FRAME-BY-FRAME again. For typical animation with 30 fps, it took me nearly an hour to insert 2 seconds animation. Maybe there is a such interface but I don¡¦t know.