UsingBlender to Achieve Believable CGI/Live Action Interaction
byJ. Daniel Sawyer
a.k.a. Lokmer

Blending CGI with Live Action via a compositing program (i.e.MainVision, After Effects, et. al.) is a time honored tradition bothin professional films and in amateur films. The process issimple: Render your CG to an alpha-enabled format, then paste thealpha onto your video clip in your compositing software using onlythe Alpha Channel. This simple technique is widely used and hasonly one major drawback: the CG characters (or ships orwhatever) cannot go behind the live action characters. Ofcourse, there is a simple workaround for this: Shoot your livefootage against a bluescreen, key out the blue, and composite yourlive footage on top of your CG footage.

However, this is not always a viable option. Witness forexample the now-infamous Jabba the Hutt scene added into Star Wars: ANew Hope for the Special Edition of the movie. In that case wehave 20 year old footage, shot on a set in (simulated) broaddaylight, with narry a bluescreen in sight. Having Han walkbehind the computer-generated Jabba is no big deal - simple alphacompositing as described above can accomplish that. However,for the majority of the scene Han is walking IN FRONT of the CGIcharacter - seamlessly, with no bluescreen. How did ILM pull itoff?
A study of the featurette at thebeginning of the Special Edition of ANH (1997 video release) revealsthat ILM created and animated a "blank" CGI Han, whichmoved exactly as Han did in the original footage. This blankwas used to mask out pieces of the CG Jabba, so that the two could beseamlessly blended without a bluescreen. This is the effect wewill accomplish using nothing more than Blender (advantages of usingcompositing software will be discussed at the end).

This tutorial assumes a working knowledge of Blender. If youfind that I'm going through some basic procedures (such as loadingpictures and videos) too fast for you, head on over to B@rt'splace and work your way through some of B@rt's excellent beginnertutorials. In order to do this tutorial successfully, you willneed to know how to create curves and add vertices to them, how tonavigate in the 3D window, how to import objects from other .blendfiles, how to change file types and set up a render, and at least thebasics of lighting a scene in Blender.

Here is the shot we will be blending (this is a screen grab from anearly saber effect test for Battle For the Zone). Without usingany compositing software, we will blend in a camera droid behind theactor and then have the actor cast a shadow on it that matches theambiant lighting.

To start with, open up Blender and delete the default plane. Now, load the picture (or video) you will be working with into theworkspace. To do this, hit SHIFT+F7, toggle down the"Background Pic" button. Hit load and load yourpicture from your hard disk. Sidenote: If you want to load a video(which, given the basic thrust of this tutorial seems more likely),select a texture from the button next to the "select texture"text and load your video accordingly.

Setup the movie texture as below. Make sure you use

uncompressedAVI (Blender won't read anything else).

Setthe number of Frames your video clip is in the

"Frames"box, and set the start frame in the "Start Fr"

box:

Forthose of you who are wondering, yes, this is a different clip thanthe one shown above - because I don't have a video of the one shownabove.

Returnto the 3D window and you will find your video or pictureloaded into the background.

Nowcomes the real work. You must construct a Blank that will serveas your alpha mask. There are a few ways to do this. Thefirst and most obvious way is to use Berzier curves to create a 2Dblank from the Camera's Point of view (NUMPAD 0). When usingblanks in this manner it is always imperitive that you work from theCamera's point of view, since this will give you the most accurateanalogue to what your footage will eventually look like.

HereI added a Berzier curve and proceeded to trace the outline of theactor's body and clothing by adding points and tweaking the curves. Since I only needed the head and shoulders to block the CGI droid Idropped in later, I only traced what was needed. Continuetracing until you are done with your blank. When done, closethe curve by hitting C. (BTW, ignore the sphere - I wound upnot using it). Note: If you're going to be using thistechnique in a video (and you want your blank to be moving) I highlyrecommend using a mesh of a human form and animating it using IKAs. Trying to animate a flat blank using Relative Vertex Keys and keepingit looking proper will prove extremely difficult. I am using aflat blank here for illustration purposes only - use a true 3D modeland animate it for video work.

Onceyou have completed and closed your curve (or imported your mesh),exit edit mode (TAB) and open up the Material Buttons window (clickon the little red ball button). Deselect the "Traceable"and "Shadow" buttons and depress the "Only Shadow"button, like so:

Now,go into the RenderButtons Window (the little button with the pictureon it). Set the file type as TGA and the color type as RGBA. If you're rendering a single static frame, load the picture you'vebeen rotoscoping from into the Backbuff. If you were to renderright now (with a picture in the backbuff) you would see a black holewhere your blank is (which is as it should be).

Timeto import your CG character. Hit SHIFT+F1, select the .blendfile to import from, and import the object(s) which you wish to use. Set them in the scene behind your blank, and be sure to light yourscene properly so that the light on the model matches the light fromthe scene into which you wish to composite it. Do anothertest render. Using a static backbuf, here's what I get:

In the above picture the droid blends adequately into the scene,however you can see by the shadow lines on the actor that he shouldbe casting a shadow back on to the droid. This is easily fixed(assuming you have your lights set up properly in Blender). Gointo the RenderButtons window and depress the large "Shadows"button, and then go into the MaterialButtons window and re-depressShadow and Traceable.

Andthere you go. Here's a picture of the final render.

Finalthoughts: This tutorial has been a demonstration of the techniqueof blank-making with the material buttons. To properly use thistechnique in video, you'll need to 1) Use a mesh of a human andanimate it properly, 2) Render your animation to a series of RGBATGAs, and 3) Put them together with the video in a compositingprogram such as MainVision or After Effects.

Pleasesend comments on the tutorial to lokmer@homestead.com or post on the Blenderwars forum at http://blenderwars.homestead.com