Here is a full resolution frame-capture of a video stream shot with the HV20 (the image is a link to the full resolution image). All of the images posted here are compressed with jpeg 80% quality, and should have most of their original attributes preserved. Note that the pulldown has been removed with After Effects, and that this was originally a frame comprised of two interlaced fields. Note that this picture is HUGE (1920×1080), so if you have a regular sized monitor, expect to scroll around to look at it completely.

If you look closely at the image, you can discern fragments of interlacing artifacts left behind from the pulldown removal. This happens because the mpeg2 compression of the interlaced image results in fields that are imperfect with macroblocks and other imperfections. This confuses After Effects’ algorithms, leaving behind chunks of image that still have interlaced areas. Effectively this just looks like bits of the image have weird blocks of ‘liney’ areas. Below is a 4:1 crop of the above original frame, doubled in size.

You can notice these artifacts on the edge of the finger. In After Effects, there are plugins to process a frame to remove interlacing artifacts. With one of those applied, the image looks a little cleaner.

There are still noticeable artifacts from the mpeg2 compression. You can make out macroblocking and chroma abnormalities on the thumb and surrounding areas. If you were to look at the individual color channels, this would be more noticeable. Usually the compression artifacts are worse in the blue channel. There is a plugin that comes with the Magic Bullet Suite called the “Deartifactor” that takes a whack at removing compression artifacts such at as these.

Deartifacted:

Non-Deartifacted – Blue Channel Only:

Deartifacted – Blue Channel Only:

Sometimes it is hard to see what the deartifactor is doing in areas of detail. Here is another comparison in another less detailed area of the image.

Non-Deartifacted – Blue Channel Only:

Deartifacted – Blue Channel Only:

There is also a filter in After Effects called Remove Grain that was once sold as Grain Surgery by Visual Infinity, but is now included for free with After Effects. This plugin is quite amazingly good at removing grain while preserving detail in the image. While the compression artifacting is not quite grain, it does do a fair amount to improve the apparent image clarity, and while it does soften the image some, it can be adjusted for good results.

Remove Grain filter applied:

Here is the final processed image, in full resolution.

Most likely the effective qualitative differences in the image when viewed full-resolution are relatively minor. However, when you get into things like doing dramatic color correction or other processing, or attempting to extract a matte from an image using color keying, it is absolutely essential to have the best quality image that you can to work with. Of course it is not really a great idea to try to use HDV for something requiring excellent chroma key matte extraction, but for 800 dollars, this camera produces some amazing images, and you would be hard pressed to find a better solution for less than several thousand. There is also the possibility of capturing 4:2:2 uncompressed video before the MPEG2 compression stage from the HDMI output of this camera, if you have a Black Magic Intensity card, or an equivalent HDMI capture device, which could theoretically be plugged into a laptop. There has been some experimentation in this area, but little noticeable benefit is likely to be achieved.

As free alternatives for the post-processing of HDV, there is a plethora of possibility with AviSynth and various other freeware windows applications. There is quite a bit of activity around the area of using AviSynth for post-processing of HV20 footage.
The Farnsworth plus Sillyman Process
Automated 24p pulldown workflow using DGIndex and VirtualDubMod
Morgan MJPEG2000 as an alternative HD Intermediate Codec
AVISynth and VDub Templates / Code

To end this post, here are some other full-frame image-captures from the HV20, with no processing applied to them other than pulldown removal and deinterlacing.

An indoor image recorded in low light, showing the characteristic of the camera in high-gain mode, which can be avoided if operated properly.

An outdoor image from the camera on a relatively bright day.

This summer was exceedingly much more busy than last summer, during which time I was unemployed and did not accomplish a great deal. This summer, I was fortunate enough to acquire workstudy, and because of that I was able to get a job working for the Electronic Media department of Evergreen. This job paid $10.78 per hour, and was for 40 hours per week. I was able to do a great deal of fun and interesting tasks, including learning intermediate functionality of Autodesk Maya 8.5, creating some enticing motion graphics in After Effects for a corporate video and some other projects, teaching a workshop on basic audio sweetening for a radio show, assisting with workshops on how to use Digital Performer and giving audio technology students proficiencies in the music technology studios, editing some exciting lectures and other events, and also doing some advanced maintenance work on the advanced music technology studio which involved rewiring the studio for installation of a 5.1 surround receiver system.

Also during the summer, I did some various work for Inventive Pictures, doing freelance grip and camera operator work. If intrigued to see visual documentation of this, look at the pictures for Pool Whore, a short shot in 720p HD about a monetarily poor girl who wants to swim in her rich neighbor’s pool.

Also this summer, I was able to go on a few adventures during spare moments. To get a feeling for what these might have been, look at the photographic documentation of them as well.

Interface
xplorer2 – The single best file manager I have ever used, and I have used alot. I wish there were anything that remotely approached an equivalent on the osX side of things. (lite version = freeware, more advanced version is shareware)
Launchy / Colibri – Keyboard launching utilities are the single greatest enhancement in interface technology since … (freeware)
TaskSwitchXP – A superior replacement for the Alt-Tab task switcher built into windows
Stickies – Sticky note program, freeware
EssentialPIM – Calendar / ToDo / Personal Organization utility, freeware

MultiMedia
Foobar2000 – Audio player, encoder, ripper, tagger, cataloger, with many more advanced features (freeware)
VLC – Monolithic multimedia player with Matroska support (freeware)
Mediaplayer Classic – DirectShow multimedia player. Best when combined with the Combined Community Codec Pack (freeware)
XnView – Photo viewer / editor (freeware)
wScite / Notepad++ – Text editors for programming and just plain text editing (freeware)

Network
Firefox / Opera – Web browsers (freeware)
Pidgin (formerly Gaim) / Miranda – Instant messaging multi-protocol clients (freeware)
X-Chat 2 – IRC client (freeware)
uTorrent – Torrent client (freeware)
StrongDC++ – Direct Connect client (freeware)
eMule – eDonkey client (freeware)

Security/System
Sygate Personal Firewall Pro – Firewall
NOD32 Antivirus – Antivirus software

[See post to watch Flash video]

Currently (Week 9), I am working on Unicorns stuff. We had a pixilation shoot in the sheep meadow behind Morgan’s house on Thursday, and we have been working on the new edit. The new edit is a revised version of the Trouble With Unicorns script that we all wrote during Winter quarter. The storyline has been altered to accommodate the footage that we have. The total length will ideally be somewhere around 20-25 minutes now, and the message that we originally intended to come across, will hopefully now come across in a more condensed but equally powerful way.

I have been working today on the Dan special effects shot, and in the TUTORIAL SCREENCAST below, there is commencement with a detailing of a novel technique of rotoscoping, and some other various happy things regarding my workflow.

(via vfxtalk.com)

This past week was an interesting one. Somehow I managed to read the entire 2000+ page Akira manga during spare time I had. It was an amazing epic journey and a far better way to escape and procrastinate school work than bad TV (though I haven’t been down that road in awhile). I couldn’t put it down. Well, I read the entire thing on my computer, so I guess I couldn’t really “put it down” in the physical sense. Sometimes I feel like everything exciting in my life is mediated by this computer. I spend most of my spare time here, sitting in this chair, interfacing with information.

Of course I start off this post saying I am behind. Then I say “well, … yeah… I read 2000 pages of manga instead of working.” Let me follow that up with some reassuring statements of what I actually did accomplish this last week.

I made some good progress on my Hybrid Music project. In Hybrid everyone in the class is working on a live performance of electro-acoustic music to be performed at a public concert during week 9. I learned how to program the drumKAT midi drum triggering device. I have created a rather decent Kontakt percussion instrument from Buchla 200 samples, and am thinking about how to actually create the rhythmic elements of my project in a live setting. Most likely it will end up being a primary bed of pre-sequenced polyrhythmic drum programming, with an additional flare of articulative live-triggered fills.

I did some great reading of theory for my New Media class, accidentally typing a 1000 word comment about the nature of language and the representation of ideas, and its relationship to the ill-defined concept of artificial intelligence. Through this discussion I was turned on to the fascinating theory of the Linguistic Relativity (Sapir-Whorf) Hypothesis, and was inspired to search for some better definitions of just what the definition of artificial intelligence really is.
On Friday, in the morning I did the aforementioned drumKAT experimentation, and in the afternoon I taught a section of about 10-15 Mediaworks students Adobe After Effects. Yes, I did stay up about 4.5 hours into the morning, that time directly cutting into my healthy 8 hours of sleep that I seldom get, revisiting previous After Effects handouts, and creating a new outline, and even attempting to bypass the necessity of live improvisation of verbiage by recording video tutorials of everything I was intending to say. Unfortunately, 4 hours wasn’t quite enough time to both figure out how to do this, and successfully pull it off, so I ended up being all with the talking. This actually went surprisingly well. I am wondering just how much the formation of concepts are tied to verbal language. I get this huge increase in my ability to think cogently and express myself successfully after every time I do one of these verbal-heavy presentations. And also, to a lesser extent, after I write an intensive paper. Needless to say, the workshop went the best of any that I have done so far, and I am really glad I had such an opportunity to share my knowledge with a group of people who might very well benefit from using it.

At 6:45am the next day (Saturday), I woke up and drove with Brad Hutchinson and Katie Gregg to Portland, OR. Here I was headed to help Brad set up for the most recent performance of his 3-projector 16mm direct animation film epic about the fracturing of family, “My Primary Colors”, accepted to be a part of the PDX Invitational, a competitive competition of the best experimental work of those accepted. After setting up in the morning, we had all afternoon to Jock around the land of Ports, so we drove to downtown and ate Greek food, and then went to the underside of the Burnside bridge, where we documented the glorious examples of dystopian ruin found right here in our very own country. These documentation included a 3-channel multi-perspectional recording from the edge of a freeway, and footage of a moving freight train from nearly under its wheels, as well as a multitude of photographs.

That night, Brad and me and Morgan Dusatko projected “My Primary Colors”. It went alright, except that the main film loop broke a little more than halfway through. This was somewhat disastrous, as you might imagine, but we managed to get it rethreaded and projecting again. Unfortunately the film decided it didn’t like the improvised takeup reel either, and so it spewed the film all over the floor… but it went across to the audience well enough to tie with 13 other people for 2nd place. The winner was a 70+ year-old man who made a psychedelic film visualizing microscopically the shifting and warping multi-colored patterns on the surface of soap-bubbles. He deserved it.

On the way home on I-5, I shot a great deal (about 40 minutes) of slow-shutter mode motion blurred video of lights with my Panasonic GS-150. I actually have a specific purpose for this footage. I am going to be doing a live performance of visual material to complement a live performance of audio material with my friend and classmate from Hybrid Music, Nic Zwart, this Saturday at 5:30pm at the Eagle’s Hall in Olympia. This is a concert series in benefit of the excellent Free Radio Olympia. I am working on compiling a “sample library” of good clips from this in After Effects, not just cutting things out, but adding effects and layering to make things more visually interesting. I have been wanting to do live musical performance for a long time. This is my chance, and I’m not going to pass it up.

Next week is my first work in progress presentation. Like I said at the beginning of this post, I have a lot of work to do. But there are 6 days between now and then. That is 144 hours. if I spend 30 of those hours sleeping, that is 114 hours of time to do things. If 12 hours of those are spent eating, showering, procrastinating, and transporting myself around, that is still 102 hours to accomplish things. That is 17 hours per day. A lot can happen between now and next tuesday. I will update on the journey if there is time, and let you all know how the impossible task of WRITING my AI project goes. And, subsequently, the ANIMATING of the first couple of scenes.

What impacts will technological advancements like this have on filmmaking? An increase in the visual quality of lower budget films? An increase in the conceptual quality of lower budget films? A decrease in the cost of higher budget films?

My guess is that as technology improves the resolution and perceptual quality of digital representations of moving images, we will become just as accustomed to it as we have grown accustomed to the amazing (as seen from a perspective of someone in the 1970s) quality of DV. Cost will drop, and quality will increase, and creating quality films will be just as much of a challenge as before. Fancy technology doesn’t make you a better filmmaker, even though it’s tempting to think so sometimes. Of course, that doesn’t make emerging technology any less exciting.

The next two weeks are not going to be about doing a ton more research however. They are going to be about answering the concrete questions necessary to actually start visual and auditory work on this project. What does the world portrayed in this animation actually look like? How will the presence of the Artificial Intelligence be manifested or embodied? How will the AIs experience of reality be embodied? How should a viewer experience this work of animation? What should they get from watching it? What are the larger issues that are important to be addressed with this animation?

Below are a smattering of notes from early portions of the aforementioned research.

what type of embodiment (that is, manifestation of experience of reality… — the perceived structure that one’s reality contains) would the AI’s experience of reality be mediated by?

how would knowledge be encoded? through language? through symbols?

Notes on “The Human Importance of the Intelligence Explosion” – a lecture at the Singularity SUmmit at Stanford.
intelligence is a ‘super power’ that all humans share. it is the foundation of all our creations. technological progress is a byproduct of intelligence.

the technologies of the future that will really matter, are those that act on the amplification of cognitive powers: nuerotechnology, brain-computer interfaces, and artificial intelligence.

intelligence is the most powerful force in the universe, and also the most non-understood. We know a great deal about the mind, but there are still things we don’t understand about it yet. We have found many “mysterious questions”, but no “mysterious phenomenon”.

the brain is the weak side of the brain-computer interface. human brains are not

molecular nanotechnology controlled by a super-human AI is a possibility to advance human intelligence, but perhaps it is not the best strategy for an intelligence explosion.
a better environment would the predictable and malleable one inside of a computer, where changes have known consequences.
The power of intelligence isn’t about new technology and gadgets. The power of intelligence is about ingenuity and surprise, going outside of the rules of the game.

rule number 1: do not trust your sense of how ridiculous things sound, when considering things in the context of an intelligence explosion.

There are different types of intelligence that might result.

Cheesecake fallacy: whenever an argument leaps directly from possibility to actuality, without considering the manufacturative properties necessary for the latter. ERROR: motive, not manufactive. why do something if there is no reason to? What would an AI want?

Ai’s are commonly depicted as an “ethnic stereotype” — all the same. In reality, there are a large variety of possible artificial intelligences. When we talk about AIs, we are really talking about mind in general. Is the space of possible mind-designs smaller than the space of the design of the human mind? no. and yet futurists commonly speak of how AIs will be in the future as confidently or more so than they would predict the behavior of an actual human being.

a self improving mind can have stable motivational architecture. (it wouldn’t modify itself in a way that would alter its current motivations).

The real problem is not a predictive problem of determining what superhuman AIs might do. It is a question of engineering. One must know enough about the problem in order to “reach” into the possible mind-design space outside of a human mind and pull one out that is a desired type… carefully, in order to achieve some measure of the desired effect.
then, making the right choice in proposing a solution to the problem of AI creation, is the most challenging problem.
(a group that is trying to solve a very difficult problem that holds out on suggesting possible solutions until the problem is discussed so much that there is nothing left to say but possible solutions, have better solutions than those that suggest solutions right away)

Ben Goertzel: “Artificial Intelligence: Now Is The Time”
most types of technological advancement have positive possibilities and negative possibilities. how we choose to use this technology is a result of the wielding of our own powers of judgement, linked to our sense of human wisdom: what would be better, as chosen by our own sense of motivation and morality?

“Victim of the Brain,” based on The Mind’s I

Computers are a medium in which we can Model anything we want to. An approach to artificial intelligence is the modeling of the functioning of human thought – the human mind. This has the effect of allowing us to understand our own functioning better. …
is the algorithmic modeling of consciousness / intelligence a valid way of creating AI?
Central attribute of human intelligence: to take an unfamiliar situation and to immediately get at what the essential elements of that situation are, and even though you might have never been in that situation before, and to ignore the silly superficial aspects of it and get right to the deeper significance, = ability.

Licklider’s concept of man-computer symbiosis, while perhaps seeming exceedingly outlandish when it was authored, is of particular applicability in today’s world. In our day to day operations, we humans (that have access to such technology) rely constantly on computational devices. Cell-phones facilitate instant and extremely mobile auditory communication with any other similarly enabled person. The Internet provides a global web of information, contributed by users, and easily searchable. Daily, we rely on the organizational and computational abilities of computing machines to assist us in our work; ubiquitous computing is becoming more and more of a reality. As Licklider so perceptively made light of, “Computing machines can do readily, well, and rapidly many things that are difficult or impossible for man, and men can do readily and well, though not rapidly, many things that are difficult or impossible for computers” (77,col1).

It seems however, that Licklider’s observations about the relational dynamics of humans and machines then stand remarkably similar to the way they are today. He noted that unbalanced relationships of dependence between man and machine were much more common than the interdependent and co-existent relationship he envisioned as being quantifiable by the term ‘symbiotic’ (74,col2). And still in 2007, the majority of human-computer relationships are heavily canted in either one direction or the other. We depend on computers and use them daily as tools. Some people have the specific job of maintaining computers, and keeping them functioning correctly. Relationships where the two exist in relative equality of function, however, are rare, if occurring at all. And it is as this point, as Licklider makes clear, that the fullest potential of advancement of our cognitive abilities becomes possible (76,col1). While to a certain extent the more advanced computational devices present today do aide us in the everyday simplification of tasks, as Licklider imagined, by means of their superior abilities of calculation, organization, and sheer processing power, we have not achieved by any means the idealized state of ‘symbiosis’ that he refers to. In simpler terms, we are significantly closer to some sort of human-computer symbiosis than when Licklider wrote this text, but we have not achieved it yet.

In Licklider’s dated discussion of the technological challenges of achieving man-computer symbiosis, circa 1960, he hits upon a few interesting points which stand solidly on two legs today. He writes that one of the most important problems in human-computer symbiosis is that of translation. How can two entities be cognitively symbiotic if they cannot communicate effortlessly? (79,col1). While he talks of recent advancements in programming languages as an example, this concept could easily be extended (in the context of today’s paradigm of computer-user interaction) to the concept of interface. The interface is the mechanism by which we as users communicate with a computer. We input information and commands into it, and receive output information back from it, in the form of visual output, auditory output, or processing of information. This is the essential essence of what computers are really useful for. But what devices are we still using for this medium of interaction so paramount in the dynamic of our relationship? The same devices we have been using for the last 30 or so years, the keyboard, mouse, and monitor. Though these mechanisms do provide a way for us to overcome the speed and language differences between man and computer, they are not a very elegant way of doing it, especially when seen in the context of Licklider’s idealized vision of ultimate communication and co-existing cognitive interaction.

We have made some progress in the improvement of interface, but when compared to how fast hardware speed and software complexity are advancing, the pace is truly glacial. Some recent advances do hold promise; with the integration of display device and input device present in multi-touch display devices, an increased efficiency and intuitiveness of operation might be achieved. This might bring us slightly closer to the incentive or goal-based interactivity models that Licklider talks about (79,col2).

The true paramount of human-computer interaction is one that has by no means been achieved yet, but one which is hinted at in recent technological developments. The ultimate communication with a computer would be one which would occur at the speed of thought. This sort of direct-mind interface would bypass the need for input/output devices, because there would be no need to be a medium between the output of the computer and the input of the brain, and vice-versa. Such concepts have been explored in a great deal of science fiction, especially in the cyberpunk sub-genre of work. The novels of Neal Stephenson, and William Gibson, as well as such works of Japanese anime such as Ghost in the Shell, deal directly with the sociological and phenomenological implications of this sort of mind-integrated computational interface, and its effects on the amplification of human powers of cognition, and even physical abilities through the usage of cybernetic technology.

With recent advancements in the area of brain-computer interfacing, this seemingly outlandish possibility of direct cognitive interaction with a computer does not seem that far off in the grand scope of technological progress. Indeed, already advancements have been made enabling animals to control robotic limbs with their minds, and enabling the brain-mediated capture of images perceived by a living cat. As these initial experiments proceed into concrete established practice and commercialization, there is no doubt as to where it will lead. The augmentation of human facilities that this will bring about, both in terms of cognitive power and in the physical augmentation through cybernetics, at least a definite contributing factor to the development of a Technological Singularity.

Maywa Denki are a group of artists and engineers in Tokyo, Japan. They are known for their creation of absurdly creative “nonsense machines“, and other works of electromechanical devices which are gloriously surreal in their purpose and functioning.

Here are some blog posts for further reading.

The Nonsense Machines of Maywa Denki
PopGadget – Maywa Denki
Suicide Bots – Maywa Denki
Hand Circus – Maywa Denki
Maywa Denki Homepage