Wednesday, January 15, 2014

EBook Promo Video - Recent Advances Towards Improved Phytoremediation of Heavy Metal Pollution

As you may have noticed from the lack of posts on this blog (and commits/bugfixes in Blender), I've been somewhat busy working on various projects over the past few weeks. Last night I finished work on my main project over that time: a promotional video for dad's new e-book, Recent Advances Towards Improved Phytoremediation of Heavy Metal Pollution.

Technical Details
  • Blender 2.69 - Windows 64-Bit Official Build
  • 100% rendered using Blender Internal
  • Produced and rendered entirely on my HP laptop - Intel Quad-Core i7-4700MQ, NVidia GeForce 470M, 12 GB RAM
  • Total: 3360 frames at 24fps - 2:20 min
  • Assembled, managed, and exported using Blender Sequencer
    • Encoded/Exported from Blender using MPEG setting, MPEG-4 format, 24300k bitrate (min=0, max=30000), AAC Audio codec.
    • Re-encoded in Windows Movie Maker, using "High Definition Display" option so that it works correctly in QuickTime player and Windows Media Player (in addition to VLC/FFMPEG)
  • Audio recorded using Audacity 2.0.4, using laptop's built-in stereo microphone, and saved in OGG format.
  • Paint.Net used for additional image wrangling, though the majority of the hand-painted texturing work was done in Blender.

I was responsible for almost the entire production - ranging from audio and visual production, scripting/planning/direction/editing, and 3D content creation (modelling, shading, lighting, animating, fx, rendering). However, a few additional assets were used to save time/better quality than what I could achieve:
  • CG Cookie Inc - CG Cookie Flex Rig 1.2.1 (released under CC-BY 3.0 Unported) - "Marvin" was used as the base character for the boy character
  • CG Cookie Inc - "Eat Sheep Collection: Farm" (released under CC-BY-SA 3.0) - Tractor model seen in opening scene
  • - Several textures, including the cracked-earth, oil drum, toxic warning sign, and striped danger markings

This is perhaps the longest and most complex Blender/video project I've undertaken to date. Unlike my previous efforts, this features many different elements including some 13 different shots (6 of which are static - mainly out of necessity). Compounding matters a bit was the fact that it's been a while since I've done any serious Blender work - especially modelling/shading/lighting/fx stuff.

Tech Setup
Originally I started working on this project using my own builds of the latest development versions of Blender. However, after enduring several days of crashes/random bugs - some due to recent WIP changes in Blender (e.g. dependency graph threading stuff causing 3d text to randomly crash, become corrupted, or fail to load) and others due to underlying bugs in the libraries used for the build platform on my machine (namely, the pthread bugs on mingw64 - which mainly manifested in crashes when using small brush sizes for texture painting in the 3D view), I ended up downloading the official 64-bit builds of 2.69 from (the first time I've ended up actually doing so in some 7+ years).

As for rendering, this done entirely with the Blender Internal renderer. Sure, Cycles is the new hotness, and makes certain types of shaders and look styles easy to set up (e.g. the opening shot - in the "cg-diffuse-clay" style). However, after performing a few tests, I also found that it was quite resource-hungry on my machine: within 30 seconds the fans would reach mid-high gear (usually only reached after compiling 60-70% of Blender's codebase), and 5-10 seconds later it would reach high gear. The more serious implications of this though were that it was also generating significant amounts of heat: given that 50 deg C is low-mid fan, 60 is mid-high fan, and 70+ is high  (IIRC, 80 deg C or so was some kind of heat threshold when things would start getting bad). This was enough the make the surface of the laptop hot to the touch and the patch of table beside it to get seriously warm.

The other reason why I ended up going with BI was the fact that I managed to get "personally acceptable" rendertimes. That is, most of the simpler shots took 1-2 seconds per frame, with the more complex ones topping out at around 10 seconds. Now, for many CG artists, such times would probably seem quite laughable - what, with the average shot in Blender productions taking at least 3 minutes to render and more complex ones taking 30-50 mins, and those in major Hollywood productions taking hours if not days. Sure, with longer rendertimes, you can milk more lighting effects and complexity out of your images. But sometimes (as in my case), it's better to keep things pared down to avoid workload <-> quality control issues.

Visual Style
Overall, I'm quite pleased with the visual style of the end product, as I managed to get results which quite closely matched what I had in mind: namely, a blend of clearly "soft CG" looking parts (i.e. the opening shot), combined with some elements exhibiting a bit of richness and gloss where applicable (i.e. the seedling - particularly the leaves). To clarify, the "soft CG" look I'm referring to here is inspired by things like Geri's Game (which really introduced me to Pixar, after seeing it frequently being played on the TV's at Harvey Norman - one of the larger computer+electrical stores here - when I was a kid), and the Chipotle ads (notably the "Back to Basics" and "The Scarecrow").

Production Workflow
In the beginning, there was an outline of the content which needed to be in the video.

From this, I started sketching out storyboards for various sequences which could visually tell the story - important since videos aren't always watched with the sound turned on! Actually, I sketched out multiple versions of these sequences, though the fundamental structure/premise of each essentially remained the same. However, lacking an overall framework/structure for the video, it was difficult to gauge the amount of time available for each section, and hence what would go into making each.

At this point, I set about properly trying to clobber together a script, and to try to get a scratch track of this recorded. This was recorded in separate parts (3-4 to be precise) using my laptop's builtin microphone, and Audacity to manage the recording process. Ultimately, it was the third set of script + recordings that was used. It turns out that recording narrative tracks is bloody difficult - especially trying to keep it all under a fixed time limit (3 min) and terminology that you're not that used to using!

Personally, I found that once I had the audio tracks recorded, chosen, and laid down in Blender's sequencer, that I was in a much better position to go through the storyboards and figure out how long things are supposed to be. Perhaps one of the first things to do once you've got the audio tracks laid down, is to create some "color" strips, set them to different colours, and try to sync them against the audio so that the transitions between these feel like they're at the right times.

Also, if you have any test renders of particular shots - even if these are just a still render the key element for that shot - it's important to get those into the edit ASAP. Simply insert the single image and stretch it out to fit the amount of space available for that shot. Doing this really helps put things in perspective, allowing you to see if the timing/spacing of everything feels right or not. At this stage, it also helps to start using markers or some other suitable technique to note down any key events/punch points where the visuals should sync up with what's going on in the audio, to give things a bit of a boost.

Only when this is ready do you start to seriously produce shots. That's because, you now have a good notion of how long each shot needs to be. As far as punch points go, those can be done on a shot-by-shot basis (i.e. Just-In-Time for production to commence) instead of needing to be done all in one go. Standard 3D production workflow applies at this point. The only other notable point here is that having the full edit fleshed out, and with placeholder elements in place starts acting as a good progress tracker and motivational aid - scrubbing through the thing, you can truly sense just how much more work you need to do, helping to avoid getting bogged down in the minutiae of a few stray pixels or similar destructive perfectionism tendencies.

This leads to another benefit: As you work on each shot, it is important to check that the timing and look of what you've got works in the larger structure. This is a good counterpoint to any tendencies you may have for letting shots just slowly "spread" out longer and longer, growing soft and mushy timing. For quick iterations, render out (if there's colour/material/lighting changes involved) the shot at a reduced resolution (e.g. 25%) or create a playblast (for animation/practical effects) the shots, and get the sequencer to refresh the image sequences for that shot (after having firstly replaced the still placeholder with that strip's image sequence after the first iteration). 

Following this workflow, in the sequencer I ended up with basically three bands of strips:
1) Audio tracks at the very bottom - Some of these overlapped so that I could eliminate the silence from either end (which is necessary to prevent the start being missed because the audio recording hadn't quite started picking up correctly due to system lag, or the end being clobbered by the sound of you mashing buttons or moving to end the recording). These are at the bottom since they basically form the fabric for holding the entire video together

2) Placeholder Strips in the middle - These are in the middle so since they're only really temporary, unlike the audio tracks, and so that you can easily slip in the real shots over the top to override them (before eventually removing them)

3) Final Shots on top - These are what the audience sees at the end of the day.

Using this workflow made it much easier to burn down the shots to produce, as you pretty much know how much work you've done and how much remains. Coupled with a suitable length todo list for each session, and it's a breeze to get through this production process.

During this process, I also noted a number of workflow problems and/or things which aren't quite right in Blender now. I'm preparing a post on these which I'll put up sometime soon. There'll also be another separate post concerning FFMPEG export issues, since I think this deserves a bit more attention.

Production Tips
Audio Recording:
  • Have a fully written script before trying to record. At the very least, ensure that at least the paragraph you're trying to record is fleshed out.
  • Get over the fact that you're going to have to record things multiple times. Whether this means not stopping after the first track you didn't stumble on (things sometimes get better from that point on, or worse), or simply trying again multiple times until you get a workable recording, getting a good recording is trick (there's a reason we have professional actors and voice actors!)
  • Once you've got a first draft of your script, try actually reading it, with your recording equipment turned on, and be prepared to reword anything that keeps causing you to stumble. You may also find that it is actually far too long.  For example, I ended up needing to reword the second audio clip multiple times, since the original version was such a horrid mouthful to pronounce, that it was impossible to get a single good take of it (besides taking a good 50-60 seconds to get through). Other horrid examples include the sentence, "This is particularly useful for ... non-plant discipline based researchers", which I kept butchering as "non-plant discipline blased researchers". "Blased?!" Argh!  
  • When recording the script, DO NOT do it on a line-by-line basis. Instead, try to go far chunks as large as can be managed without causing excessively long retakes. (Alternatively, just going for chunks delimited by natural breaks in the thematic material). Unless you are a bespoke professional actor, you probably don't have enough vocal control to make it all sound like a coherent whole in the end, instead of little sentence-long sound bytes.  

Production Organisation:
  • Use your non-linear video editing software as your "live edit" or dashboard for the project. This is where you collect together the current state of the project, allowing you to get an overview of your progress.
  • Besides acting as a dashboard/todo list, the live edit also helps you make sure that various shots don't start going pear-shaped timing wise. That is, use the live edit for checking that all timings remain on track.
  • Note "key events" in the audio which should be reflected in the visual elements. Then, transfer the relative timings of those into the actual shot files to ensure that you get a nice sync.
Producing Content:
  • Close enough is good enough. Unless you're really handy with video encoding codecs, chances are, any fine details you try to get into your video is just going to be clobbered over by whatever atrocities bad FFMPEG settings will cause at the end. In fact, FFMPEG export is still a huge problem area in Blender (though admittedly much better than when I last tried it several years ago), with the default preset setting still resulting in bad quality output that cannot be reliably played by many common players. More on this in another post.
  • Cheat, cheat, and cheat some more. It's the end result that counts! For example, the leaves on the seedling are actually not connected to the stalk. In fact, they actually stick out the back quite a bit. But from the front, it looks good enough - especially at the common resolutions that people will be viewing this (and for the intended audience - i.e. non CG professionals!) that non of this matters.
  • Be aware of any resource limitations - budget, time (runtime, production time, rendertime, manpower time), and personnel (i.e. number of people, skill levels in various tasks and/or any significant limitations that need to be mitigated) - and adjust the overall complexity/quality/style of the production to suit. In other words, if you know that you suck at modelling, go for simple shapes and/or avoid complex organic forms. Similarly, if your texturing is not really your thing, visual simplicity is also quite a blessing, as it lessens the expectation of visual complexity. (NOTE: is it any wonder that so many things in modern times are now tending towards minimalism ;) 

Here are some examples of some of my favourite shots from the clip.

Farm Shot
This is one of the first shots I thought of, and is perhaps the most complex in the video. It is designed to show how human activities are contributing to toxic metals polluting the environment, and the effects of this. The version shown in the final video is not even the full version of this scene that I had originally envisaged (which would have featured a few smoking factories spewing out toxic/polluted water from some side pipes), as I ended up having to cut the inclusion of those components for time limit/edit timing reasons.

From top to bottom: 1) A typical orchard, 2) with trees smothered in fruit (after various chemicals were used to artificially boost production), 3) the effects of these chemicals on the environment and the trees.

The visual style for this scene was chosen for several reasons. Perhaps the main ones were that: 1) I really love this sort of look, with everything shading using a "nearly pure diffuse" shader (with any specular components only maintained for a light sheen and/or to keep rim lighting effects) and 2) keeping the shapes simple and low-poly meant that I didn't have to work too hard to model everything. The fact that it also works quite well for clearly telling the story is a secondary effect of this.

As a bit of fun Blender trivia, despite this looking like a rather typical Cycles global illumination scene, it was entirely rendered using the BI using Ambient Occlusion instead! Not only that, but there are actually NO LIGHTS in this scene at all!

Toxic Metals Soil
The idea for this shot again came quite naturally from the outline, with the script adjusted to make it work even better. While the idea of what should happen was always quite clear, nailing a satisfactory presentation was harder. Particular complications included trying to make it look like the shot was set underground without looking tacky.

In the end, I'm quite happy with how it turned out. Especially at the end of the scene, with the entire background covered with these heavy metals. After trying various solutions for trying to uniformly cover the background like this, I ultimately ended up resorting to manually placing each of these by hand (the exact technique I use for getting collections of things to show up is discussed in another post).

The Super Seedling
This is perhaps the first shot I actually started working on (even though it wasn't the first that I came up with), since it was the first that I had a complete idea of how I wanted it to play out. Specifically, this shot combines two personal favourite visual elements/effects: that of objects coloured with a certain cluster of "rich greens" and set against a black/dark background, and backlit scenes with starburst halos.

The super seedling and a glowing halo of light around it

The Leaf - The texture here was hand-painted in Blender to capture the specific "rich and luscious greens" look I wanted. I particularly love this particular combination of colours, especially when tightly packed like this and set against a dark background.

Super Seedling in Action
Finally, late in the process, I needed to quickly put together a few other shots to finish off the video. Not wanting to spend too much extra effort modelling more elements, I ended up taking the super seedling shots (complete with halo background) and "enhancing" them with addition props to illustrate the main point of each shot.

A special titlecard (for the uploaded version) I made from the Part 1a page

And for completeness (though I personally think this one was half-baked):
The model in this one was actually repurposed from it's original purpose. It was something I modelled quickly for fun, as at the time I had anticipated doing a bit of a transition shot for the end of the video where I'd go from this 3D model fading into tree on the cover of the book (which this is actually modelled off), with the camera then backing off to reveal the entire cover of the book with "buy now" text labelled off the sides.

Repurposed Models
Several models ended up getting repurposed as the shots that they were originally going to be in got chopped from the final edit.

The tree above was originally meant for a transition shot at the end, and/or maybe to appear in the background of the farm scene. However, after all these shots ended up getting cut/abandoned, I ended up incorporating it into the above shot (as a bit of a standin). It was however also used as a good test of Cycles to test the feasibility of using it for some shots...

Cycles render of the tree - it was quite a nice render, but was too computationally intensive and slow to be feasible in the long term.

Another notable example of repurposed assets are the drums of toxic chemicals seen at the end of the farm scene. These were at one stage going to get their own shot at the start, which would then transition into a shot of factories spewing out various forms of pollution (BTW, the whole scene was going to have a light orange-brown colour cast to it too).

For various reasons, the additional decals on the other drums didn't seem to really work out after I'd fixed the linking issues I had trying to get nested linked groups of these into the farm set. Oh well...

The Boy Shot - Cookie Flex Rig
Finally, a brief word also needs to be made about the "boy" character. This was the "Marvin" character preset from the Cookie Flex Rig v.1.2.1

For that shot, I originally wanted a baby in the style of (who am I kidding... I actually wanted the actual Pixar character) of Jack Jack from The Incredibles. However, considering that that shot wasn't even a big part of the video, and that the character would only appear on screen for 2 seconds max, I figured it wasn't really worth spending a whole lot of effort trying to model the desired character specially for this.

In the end, I ended up using the Flex rig, hoping that I could eventually find/tweak things to give me something that looked close enough to what I wanted in a pinch. The Marvin character template was the closest I got. However, initially, it had some serious lack-of-appeal issues; namely, it had a tiny + pointy noise, and deep-set eyes with heavy eyebags which made him look quite ugly.

Fortunately, after playing around with the rig for a while, I discovered that a solution:
1) Simply to take the cheek controllers, and jack them up about about an inch or so. Doing so, the cheeks ride right up, covering the nasty eyebag lines, resulting in a fuller facial appearance.
2) Shrink the size of the eyes a bit, and adjust the inter-eye distance. The default eye appearance gives the characters a bit of an ugly bulging eye effect.
3) Adjust the width of the nose so that it was less pointy and MJ-ish.

TBH, I'd recommend applying similar customisations to other characters based on the Cookie Flex rig, which, while flexible and a great resource, tends to result in slightly ugly characters at times.

One other thing that should be noted here. Originally, I accidentally opened the file in trunk instead of 2.69 (which I'd been using for the rest of the production). When testing the lighting setups initially, I found that the default light setup in the file for BI resulted in harsh shadows and nasty overall light distribution. Also, the stucci texture on the skin for roughness was resulting in a really distracting/cheap 3D look.

To get a better result, I removed that texture (for the style I was going for, skin is supposed to be perfectly smooth, even if that doesn't resemble reality in any way!), and turned on + up the Ambient Occlusion. This improved the appearance a lot, but things were still a bit overexposed (too much white-ish glare on things). However, reopening the scene in 2.69 resulted in a "just right" exposure, complete with a near-perfect "cg diffuse clay" shader look that I wanted. Perfecto!

No comments:

Post a Comment