cerebular exocarp

A big part of visual effects work is removing or altering unwanted items in shots. Wires or rig, blemishes on actors or in the set design, text or signs on buildings, all of these things are prime candidates for visual effects cleanup work.

There are many possible techniques to use for cleaning up a shot, ranging in difficulty from extremely easy, to mind-numbingly complex. How hard it is depends on how complex the background behind the object being removed is, and what might be occluding the object being removed in various parts of the shot.

For example, if there is a large unfortunate piece of rig that happens to be in front of a complex and defined tree-branch blowing wildly in the wind, occluded in the foreground by a healthy wisp of smoke, cue the nightmare scenario. Basically the aim of cleanup work is to re-create the background behind the object needing to be removed, such that a person can’t tell there was ever anything there.

Here are some of the techniques used to do this.
2D or 3D tracking of still “cleanup” images into shots: this works well for background objects that are not deforming, for example, the sides of buildings, trucks, rocks, and other hard things. This technique does not work as well for soft moving things like people, clothes, energetic trees, and water. Another thing that confounds this technique is interactive lighting changes. If there is a flickering light on the side of a building, using a still image to clean up something on the wall of said building will look out of place, unless a keyframed color correction is applied to match the lighting changes.

Cloning one area of an image to another area, in order to cover something up: This works well for shots where the background of the object needing to be removed has a moving texture. For example, for something like ripples in water, still image “patching” will not work because the ripples in the water have to move. Since the texture of the water is ideally relatively consistent in its pattern of ripples, cloning from one area of the frame to the other might not be noticeable. However, if the background’s pattern is non-repeating or complex, this technique might easily be foiled.

Clone Painting: This technique is varied and quite effective with the right tool in skilled hands. It is similar to wielding the “rubber stamp” tool in Photoshop, except that it must be kept in mind at all times that one is working with a moving shot. One can clone areas from adjacent frames to replace the background over a moving wire. In order to do this effectively, the plate has to be motion tracked and stabilized to the object being manipulated, so that the object being removed doesn’t change position from frame to frame. Clone painting from the same frame using an offset to remove something on a moving object also can work well. When cloning with an offset on consecutive frames, one has to be very careful in order to avoid motion artifacts that result from the cloning happening slightly differently on each frame. A first inclination might be to just clone out an object on each frame and be done with, but when you watch it back in motion, horrible boiling artifacts will occur over the object that is removed so perfectly when looking at each frame individually. Generally, offset cloning is easier to get away with on edges, and objects in motion, and harder to get away with on static objects that have subtle gradients.

Here are a couple of video tutorials on how to accomplish some of the things discussed, using The Foundry’s Nuke 6.0.

Watch in HD: 2D Tracking and Cloning from Jed Smith on Vimeo.

A simple tutorial in Nuke on how to clone from one area of a moving image to another, using 2D tracking and stabilization, and basic compositing. Uses a shot from The Hotdog Cycle, produced by The Last Quest in Seattle.
The Hotdog Cycle Trailer

Watch in HD: Temporal Clone Painting, Patching, and Grain Manipulation from Jed Smith on Vimeo.

A demonstration of a method of cleanup using clone-painting from adjacent frames on a stabilized plate, in NukeX 6.0.
The shot used is from the animation “High Strung”, produced by Tommy Thompson at The Evergreen State College.
Tommy Thompson’s Production Blog
A Short Documentary About The Project.