Last week was Thanksgiving Break week. I was going to not take a break and spend the entire time in my room working. Then I decided that was a bad idea, and went up to see my Mom for a few days. She lives near Friday Harbor on San Juan island. I did not get nothing done while I was there, however. I brought audiovisual documentation equipment consisting of my Canon HV20 HDV camcorder, my Canon 350d digital still camera, my brother’s laptop and Tascam US-122 audio interface, a Rode NT4 stereo microphone that Peter Randlette was kind enough to lend me, and 2 MCA SP1 microphones that my brother also has, for stereophonic recordings of varied stereo width, or to have mono documentation options.

I captured a large amount of moving water textures at dusk on the ferry trip over that will be useful for the 2nd dream. I recorded audio of water lapping at the ocean shore that is a very good approximation of my memory of the 1st dream. I captured photographs of many moist rocks while at said shore that will also be useful to create the environment of the 2nd dream. I also got to direct my mother as a voice actor, while she delivered her lines from the 3rd dream. Most interesting! I also captured some brilliant wind-in-trees ambiance that will be quite useful for sound-design in the 1st dream. Here is a pointless landscape picture.

When I got back, I had a bit of a difficult time getting motivated. I took Thanksgiving off. I started then working through even more tutorials on Nuke and Maya, delving deeper into the ridiculous complexity of these programs. I discovered by happenstance, a new tool which is magically assistive for tasks involving rotoscoping. It is called Motor, and is part of a suite of tools by Imagineer Systems. What is interesting about Motor is that it uses the companies proprietary planar tracking technology to track the motion of objects in footage, and extrapolate the motion of a 2d plane in 3d space from it. This amazingly accurate track can then be applied to the motion of a rotospline. This effectively means that 80% of the roto work is done for you, and all you have to do is provide the fine-tuning keyframes. This is difficult to explain completely. Once I start roto work on all of the pornography for the elements in the 3rd dream, I promise to post a screencast and show what I am using, and how I’m using it to do what I’m doing.

Then I decided that it would be a good idea to stop burying my head in software for awhile, and start work on my project, bringing about a new paradigm of learning software as a necessity to achieve an end-goal of creating something specific! Always a better way to learn, right?

Fortunately, the first scene is not too hard, and I had a plan of how to do it. This plan involved the creation of a wide-angle still image stitched from many tone-mapped HDR exposures. I resorted to this complex plan of image creation because the picture I had remembered from my dream couldn’t really be captured with … my camera, using conventional techniques. I used the PTgui software to do the image stitching, and Photomatix to do the HDR processing and tone-mapping. This is a similar technique to the processes of gigapixel photography, and the specific innovations of these techniques in regard to visual effects, pioneered by Eric Hanson and compatriots, at XRez. I then took the footage into After Effects and did some 3d camera mapping piled on the filters until it looked right. (After Effects is more suited to this type of effects processing than Nuke, although I did play around with Nuke’s 3D system with this footage a bit).

Soundtrack:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

3 Comments

Recently I was doing an activity that I have previously found myself doing on multiple occasions: Looking through all of the different codecs that are options to render out my newly perfected 1080p23.976 scene into. I could choose the apple Animation codec, which is rumoured to be nearly lossless, at the expense of harddrive space and at the cost of being unable to play it back in real-time, but to the benefit of having a lossless image. A similar predicament with the 8-bit 4:2:2 Lossless codec. On the other end of the spectrum, there is the temptation of encoding to something like mjpeg, and having a small file, easy playback, but horrendous image degredation.

Apple and Red Cinema have been hyping the new Apple ProRes 4:2:2 codec for a while. “Perhaps I should use that!’ I thought. “If this codec can compress 4k video from the Red camera down to 200mbits/sec, then surely it could help me with my mere 1080p workflow problems!” And so then I decided to do some research about just what technology the ProRes 422 codec used and how it compared to others of similar variety.

During that process, I came across this comparison of the ProRes 422 codec and the Avid DNxHD codec, which is apparently one with a similar intended specialty of “Why yes! I can compress your HD footage so you can actually work with it, while preserving detail!” Apparently, the ProRes 422 codec is a bit more lossy at similar bitrates than Avid’s codec. After a little bit of searching, I discovered that not only are Avid’s codecs freely available, they are cross-platform between Windows and OSX. This issue was lending me trepidation earlier on as well, myself considering what would happen if I had a bunch of scenes rendered out in Apple ProRes 422, and then needed to reboot into Windows to do some specialized work on something, only to be unable to read my files!

Anyway, barring complete disaster, I will be using the Avid codecs for some of my needs.

No Comments

Here are rough storyboards for my project. If the nature of my project is unknown to you, I posted a Visual Treatment earlier.

These storyboards represent a keyframed point-to-point abstracted version of the visual contents of my project. A lot of detail is missing, such as specific transitions and even in-between points of some shots. I am still not sure about the visual treatment that I am approaching this project with. As touched upon in a reply to a comment left by my Mother, I am somewhat torn between the idea of representing the dreams as faithfully as I can recollect them, and the idea of altering the dreams to provide a coherent or more easily digestible presentation for the viewer.

Some examples : I was having a very difficult time while storyboarding the 3rd dream. There is such an emotional and personal experience element to this dream that is difficult if not impossible to convey when using the convention of 1st person perspective shots that I have decided to use in order to try and accurately represent the visual experience of the dream. A purely audiovisual format is a much much lower resolution type of medium than a real experience of dream or of reality, in the sense that much less can be communicated and be transparent to feeling. I think that I need some way of bolstering that missing information of emotional poignancy, so that it is still present in the work. The challenge is how to do this without completely altering the form of the dream as it was originally experienced. I am unsure of what to do, and am considering whether it is more important to convey the original visual experience of the dream as accurately as possible, or to depart slightly from the original visual experience and in doing so be enabled to better convey the emotional aspect of the dream as it was originally experienced.

At any rate, I thoroughly consider these storyboards to be a starting point, and a place to create changes and build from. A definitely and incrementally defined structure of how this project will come to be, they are not. Nor could they be with my obviously lacking drawing ability which, though getting slightly better, is still sorely in need of bettering. I feel like this is a valuable first step in starting to create the images themselves though, and will work to my benefit in not getting lost in the process of production.

No Comments

Fallen Art is Tomek Baginski’s 2nd short work of 3D animation, after Cathedral. It is really quite amazing in its organic detail and ambiance. You don’t see 3D work as good as this very often. Below are embedded Stage6 (requires stage6 plugin, but is very high quality) and Daily Motion copies of it. You can also buy a dvd.

1 Comment

In a previous post, I briefly discussed some of the specific concerns of working with footage from the Canon HV20. In this post, I will talk about these things in a little more depth, and with some example pictures to more fully demonstrate what I’m talking about.

Here is a full resolution frame-capture of a video stream shot with the HV20 (the image is a link to the full resolution image). All of the images posted here are compressed with jpeg 80% quality, and should have most of their original attributes preserved. Note that the pulldown has been removed with After Effects, and that this was originally a frame comprised of two interlaced fields. Note that this picture is HUGE (1920×1080), so if you have a regular sized monitor, expect to scroll around to look at it completely.

If you look closely at the image, you can discern fragments of interlacing artifacts left behind from the pulldown removal. This happens because the mpeg2 compression of the interlaced image results in fields that are imperfect with macroblocks and other imperfections. This confuses After Effects’ algorithms, leaving behind chunks of image that still have interlaced areas. Effectively this just looks like bits of the image have weird blocks of ‘liney’ areas. Below is a 4:1 crop of the above original frame, doubled in size.

You can notice these artifacts on the edge of the finger. In After Effects, there are plugins to process a frame to remove interlacing artifacts. With one of those applied, the image looks a little cleaner.

There are still noticeable artifacts from the mpeg2 compression. You can make out macroblocking and chroma abnormalities on the thumb and surrounding areas. If you were to look at the individual color channels, this would be more noticeable. Usually the compression artifacts are worse in the blue channel. There is a plugin that comes with the Magic Bullet Suite called the “Deartifactor” that takes a whack at removing compression artifacts such at as these.

Deartifacted:

Non-Deartifacted – Blue Channel Only:

Deartifacted – Blue Channel Only:

Sometimes it is hard to see what the deartifactor is doing in areas of detail. Here is another comparison in another less detailed area of the image.

Non-Deartifacted – Blue Channel Only:

Deartifacted – Blue Channel Only:

There is also a filter in After Effects called Remove Grain that was once sold as Grain Surgery by Visual Infinity, but is now included for free with After Effects. This plugin is quite amazingly good at removing grain while preserving detail in the image. While the compression artifacting is not quite grain, it does do a fair amount to improve the apparent image clarity, and while it does soften the image some, it can be adjusted for good results.

Remove Grain filter applied:

Here is the final processed image, in full resolution.

Most likely the effective qualitative differences in the image when viewed full-resolution are relatively minor. However, when you get into things like doing dramatic color correction or other processing, or attempting to extract a matte from an image using color keying, it is absolutely essential to have the best quality image that you can to work with. Of course it is not really a great idea to try to use HDV for something requiring excellent chroma key matte extraction, but for 800 dollars, this camera produces some amazing images, and you would be hard pressed to find a better solution for less than several thousand. There is also the possibility of capturing 4:2:2 uncompressed video before the MPEG2 compression stage from the HDMI output of this camera, if you have a Black Magic Intensity card, or an equivalent HDMI capture device, which could theoretically be plugged into a laptop. There has been some experimentation in this area, but little noticeable benefit is likely to be achieved.

As free alternatives for the post-processing of HDV, there is a plethora of possibility with AviSynth and various other freeware windows applications. There is quite a bit of activity around the area of using AviSynth for post-processing of HV20 footage.
The Farnsworth plus Sillyman Process
Automated 24p pulldown workflow using DGIndex and VirtualDubMod
Morgan MJPEG2000 as an alternative HD Intermediate Codec
AVISynth and VDub Templates / Code

To end this post, here are some other full-frame image-captures from the HV20, with no processing applied to them other than pulldown removal and deinterlacing.

An indoor image recorded in low light, showing the characteristic of the camera in high-gain mode, which can be avoided if operated properly.

An outdoor image from the camera on a relatively bright day.

No Comments

I have been working on storyboards for quite a while now. It is a very challenging process for me to interpret the visual treatment that I wrote and try to translate the image that I have in my head of what these things will look like into a rough image on paper that somehow conveys the image in my head. I find myself often erasing, realizing that what I just put to paper is nothing like what I imagine. Tedious process aside, here is an example of the work I am doing, for the interest of anyone reading.

The following image is a few panels from the first few shots of the first dream.

2 Comments

Among the constant plethora of activities in the area of technical matters, (that I sometimes seem to use as a form of procrastination from ‘real’ work), I have been doing some interesting things.

I have been going through the manual and tutorials on Nuke, and am quite struck by its differences from After Effects. Nuke is less friendly in the construction of the user interface, and yet very powerful. Like many compositing applications, it features a node-based interface. A node-based interface is basically a different way of combining different elements into a final image. An image or moving image is loaded in, and then multiple nodes are linked together in a stream, each applying some process to the image. This combination of nodes forms a ’script’, which is much the same as a project in After Effects. After Effects features a layer-based interface, where different elements are stacked on top of each other in layers, and the layers are arranged in time. This is what I am used to, but a node-based interface is beginning to make sense for some more complex projects using many image elements.

I have also been working through some more advanced tutorials on texturing and modeling in Maya, attempting to build on what I learned this summer, and gain the knowledge I will need to actually create some of the crazy things I have in mind for this project.

Close to the end of Week 06, I received a new HD video camera, the Canon HV20. Purchased from Newegg.com for $799, it is capable of shooting 1080p24 in the HDV format. I will be using this to acquire textures and live action image elements for my project at a high enough resolution to successfully work at 1080p24 for the final output. Of course with a new acquisition format, there is a great deal of testing and learning to do in order to become comfortable with a post-production workflow.

Because the HV20 is capable of recording in 23.976 progressive frames per second, and the HDV format specification calls for interlaced 29.97 frames per second material, the HV20 has to do some processing to record its beautiful progressive data to an HDV tape. Similar to the Panasonic DVX100, which records 23.976 progressive fps standard definition material in a 29.97 interlaced fps video stream by applying a pulldown to the footage, the HV20 records a 1080p24 stream of video, but packaged in a 1080i30 video stream in order to conform to the HDV standard. The fact that editing system support for the HV20’s footage is somewhat spotty that a process of removing the pulldown outputting to an intermediate codec is a necessary step. Adding additional complication is the image quality of the HDV format itself. Since the format uses mpeg2 compression, there is a relatively significant amount of image artifacting compared to DV. A step that many experimenters with this camera have been using to improve this situation is to add an artifact-removal step to the process. This way, the original footage from the camera can be processed from it’s original mpeg2 format, having its pulldown removed to achieve progressive frames at 23.976 fps, having compression artifacts filtered to improve the image quality, and then outputting the footage to a relatively lossless intermediate codec for editing and post production work (such as mjpeg2000, apple intermediate, pro res 422, or uncompressed). Of course with the adoption of HD intermediate codecs, more difficulties arise with storage space because of the extremely large size of uncompressed HD video, playback issues because of the extremely large data throughput needed to play uncompressed HD video (and the relatively limited transfer rates of hard drives), and so forth. For a large documentary, it would likely make the most sense to edit the footage in the acquired format of HDV and trust your editing system to deal with mid-GOP cutting recompression issues in an intelligent fashion. For this type of project however, because it is more animation and effects oriented, it makes more sense to have uncompressed files to work with… or at least I think it does.

On the bright side, the images from this camera look simply amazing, and I am very excited to incorporate this technology into my project, and figure out what will be the least painful method of working with the footage. Of course my primary goal is to make a good work of media, not to get caught up in the technology to the exclusion of all else, but I don’t think I am falling off of that cliff quite entirely yet.

1 Comment

I have been neglecting the blog rather badly these last couple of weeks. In order to compensate for that, I will regurgitate all of the pent up matters that have been needing some writing to be done about them, and with posthaste.

The current week of Fall Quarter is Week 07. Last week (Week 06) I had an exceedingly bad group of days, and did not accomplish very much. Accordingly, I missed my meeting with Sally, because I had nothing to show to her. I have gotten back on my feet though, and have been making progress.

During Week 05, a re-consideration of the feasibility of accomplishing this project in one quarter was undertaken, and it was decided that I would continue this project into Winter quarter. A revised draft of my production schedule is available, and it will hopefully be reasonable enough as to be possible to accomplish. Also, the first dream was discarded and another one was intentionally selected in order to bring about a better flow in the larger structure of the project. This does violate my initial adherence to a process of random selection, but I value the final product more than a strict formal process. The Revised Visual Treatment reflects this change, and has been significantly revised to be more clearly written in the sections that describe the other dreams as well.

No Comments