[MusicViz Project] Part 3 - Breakdown of FFMPEG "showcqt" Experiment

This is the third installment of my ongoing series of posts on one of my long-term projects to develop a new automated technique for visualising music.

To motivate today's discussion, here is the final video clip rendered from the experimental technique being discussed in this post: 

https://www.youtube.com/watch?v=LTMpzmdI9qQ

 

And here is the command used to render that:

ffmpeg -y -i 20230803-v02.flac -filter_complex "[0:a]showcqt=s=500x1920:axis=0:cscheme=0.6|0.7|0.1|0.1|0.8|0.5,crop=500:1392:0:4000,setsar=1,transpose=2[vcqt]; [0:a]showwaves=mode=cline:s=1920x100[vs]; [vcqt][vs]overlay=y=H-100[v]" -map "[v]" -map "0:a" -c:a aac "mv_20230803.mp4"

 

Let's annotate that to show the different parts more clearly (see the breakdown following this for a rough description of what each part does):

ffmpeg -y -i 20230803-v02.flac -filter_complex "[0:a]showcqt=s=500x1920:axis=0:cscheme=0.6|0.7|0.1|0.1|0.8|0.5,crop=500:1392:0:4000,setsar=1,transpose=2[vcqt]; [0:a]showwaves=mode=cline:s=1920x100[vs]; [vcqt][vs]overlay=y=H-100[v]" -map "[v]" -map "0:a" -c:a aac "mv_20230803.mp4"

[MusicViz Project] Part 2 - Motivations + Rough Directions

This is Part 2 of what will hopefully be a series of posts documenting my attempts to build a music visualiser for automatically creating interesting dynamic visualisations for my back-catalogue of music I've been writing + recording of the past few years. Last time I checked, in total there's probably somewhere between 3 and 5 hours of "finished" or "presentable" tracks, with most averaging about 1 minute in length (most come in under that around 52-55 seconds), with only a few reaching 1:10 mins, and only 2-3 blowing out to ~2:30 mins.

Most notably, there are 2 playlists (or really "albums" by this point) of material I produced during the few months I was holed up in my room writing my thesis. During most of the day and night, I'd be listening to these playlists while slaving away in my text editor, desperately trying to make some progress (some days much more successfully than others); and then, to take a break / recharge, I'd write or record some music based on fragments that would come to mind. Rinse and repeat for several months. As my thesis grew, so too did these playlists, which each ended up over an hour long in the end.

For several years, I've been wanting to package these up in a suitable format to release into the world. Currently, only a small handful of these tracks have been heard by anyone other than myself, but certainly not the entirety of these playlists in their totality. Yes, granted, the expected audience is probably vanishingly small, as they are certainly not "mainstream", and don't fall neatly into established categories... hence, even if/when I do release these, I hardly expect many people to actually listen. Then again, if anyone's interested, I have actually since produced a few more hours of similar / evolved material since then LOL - heck, I'm listening to one of the newer playlists as I write this, and even I am surprised by some of the material I recorded even a few years ago.

[Music Viz Project] First Version of Pitch-To-Colours Mapping

After procrastinating over this for a few years, I've finally put together a first version of a mapping for the colours I typically associate with each pitch - one of the key elements for the music visualisations I've always wanted to generate for all the music I've been writing + recording over the past few years.

This is actually my second attempt at putting together such a chart. The first one (which I can't seem to find right now) was only partially complete, as at the time, I kept struggling over whether I'd picked the perfectly calibrated shades for each, which then meant I never got the basics down.

 

So without further ado, here's a rough chart:

 

 (Disclaimer: I wanted to clean it up more, but Musescore doesn't let me easily insert/delete excess notes in the middle of a line without re-entering the notes and then losing the colours. So... meh!)

Oil Column Heater Model

As a little warmup exercise for a video project I'm working on, I went ahead and modelled Mum and Dad's old oil column heater. Why a heater you may ask? Well, a few nights ago, I caught sight of the the heater's light casting a lovely red glow on the wall, nicely complementing the deep blue light coming from the windows in the darkened room, and thought to myself, "Hmm... I wonder if I could replicate this scene in Eevee..."



Of course, what's a story without a little drama ;)  The one snag in all of this is that it turns out I haven't actually modelled anything in over 2-3 years! Sure, there have been Grease Pencil tests/animations and an occasional quick test objects, but models meant to look like something in particular? Zilch  (IIRC, the last time I did a lot of modelling was on Dad's ebook promo video, which was eons ago...)   Furthermore, I've mostly focussed on organic stuff (i.e. characters), so modelling an hard-surface object was a nice little challenge.

Happy Chinese New Year 2017 - The Year of the Chicken!

Microsoft's Revamped "Paint"

This evening, it came to my attention that Microsoft has been working on an updated version of their classic "Paint" app, but "Windows 10-ified" and with support for basic 3D drawing/sketching...





Woah... this is an interesting move, though not completely unexpected, with the other basic 3D creation tool they bundle by default (IIRC) that's aimed more at people interested in 3D printing simple objects. Having begun my digital art journey with MsPaint in Windows 98 (I like to think I got quite skilled at drawing using a 2 button mouse on a pixel-by-pixel scale, "graduating" on to animate my first characters using MsPaint + MsAgent), it's interesting to see what the next generation of budding artists playing around with a "vanilla" PC will be able to start from.

GPencil V2 Has Landed

In case you've missed the news, Antonio's "Grease Pencil v2" branch landed in master earlier today. Check out the upcoming nightly builds for the new goodies :)

And now for some obligatory doodles I made while giving the tools a bit of a shake down :)

Seagulls vs Waves Clip

Keeping with the theme of "fan-art" of sorts, here's a little video I put together from footage I shot while out a Sumner beach on Saturday afternoon...



From the video's description:

A pair of seagulls hunting for food along Sumner Beach, between the receding tides. Filmed yesterday during a relaxing stroll along the shoreline out at Sumner, after a great lunch. The sea yesterday looked as I've never seen it - the water level was really high, and the waves just kept coming in thick and fast.

The scene reminded me a lot of Pixar's amazing short film "Piper" (showing in front of Finding Dory). It's one of my absolute favourite shorts! Great story, wonderful animation, amazing rendering, and OMG cuteness overload!

While this shaky phone-cam (at reduced resolution to save disk space) doesn't hold a candle to that, this clip is still very much in the same spirit!

The soundtrack here is a little track I recorded specially for this footage. It's done using the "Violin Layering" techniques I've been playing around over the past month of so - basically, I recorded the first pass of the music watching the video (via my phone), and then worked on recording extra tracks for it while keeping in mind the general mood I was going for (in addition to the key beats of the clip). It's not quite 100% there yet, but as a first attempt at doing this, I'm overall happy with the results!

Mid-Week Doodle - Blurry Tree-Lined Path at Night

Another little doodle/painting session to relax a bit, inspired by the experiments I'd been doing this autumn with deliberately defocussing my lenses to get an optimal "painterly bokeh" look. (Long-time followers of this blog may also remember that a few years ago when learning how to use GLSL, I mused about developing a "disk splatting" technique for rendering bokeh-filled scenes; this was basically another attempt at achieving similar results, except in the real world ;)

This image started by being just an attempt to test the "disk splatting" blurry-tree technique I'd been interested in trying (especially after accidentally "discovering" a way to achieve that look in Krita using a particular circular brush, playing around with different combinations of Opacity + Size settings). However, seeing the result and feeling that the result still somehow lacked a bit of the "oomph" the foliage needed, but not being able to quite pinpoint it, I started blocking in some background (in the hope that it might help), and before long, I'd ended up with this scene.

It is partially drawn from somewhat idealised memories of the annual Lantern Festival in Hagley Park (this particular setting is a bit closer to this year's layout, leading to the area where they had the fireworks). However, it is probably influenced more by the beautiful work of Leonid Afremov - he's one of my favourite painters, with his unique style chunky blocks of bold + vibrant colours, which he paints by using a little triangular knife to scrape lumps of oil-based paint onto the canvas. In that case, I guess you can probably consider this an example of some crappy "fan art" of sorts ;)

Sunday Afternoon Doodle (+ Additional Music Mashup)

Just a random doodle on a cold and dreary Sunday afternoon, playing around with Krita's Multibrush tool... It's quite bizarre using this tool, yet sometimes, you can get some funky effects (like the middle bit, which was quite captivating to draw).

"Piper" Review - Finding Dory Short

I decided to pull this one out into its own post, as it kind of deserves it IMO...



Birds. Macro Photography. Shallow DOF. Awesome Renders. Entertaining + Creative Animation and Sound Effects. Pixar. Fluff-ball Cuteness Overload!

This is one of the best little shorts I've seen. I really, really, really love it! It's just soo adorable, and features a lot of my favourite things.

Finding Dory Review (SPOILERS)

I just got back from a fun rainy afternoon watching Finding Dory  :D

Happy 16/06/16

Just a little bit of fun birdie art for a special date, featuring none other than one of my favourite little silvereyes :)

Painted using Krita

Texture Painted Cube Test - Palettes

Just a short turntable anim of a texture painted cube for fun this evening....



The story behind this:
I was trying to find a way to test the "Palettes" system in Blender (which I'd just only recently heard of). Apparently it's only for the texture/image paint stuff (?), so I went ahead and started texture painting a cube!

Lo and behold, a few random dabs of colour using my tablet later, it turns out to look pretty darned interesting! Hence the turnaround, and a quick BI render of it.

Grease Pencil Fun - "Hedgehogs" Character Sketches

I had a bit of fun this evening doing some little sketches of my "hedgehog" characters using Grease Pencil (with Antonio's stroke quality tweaks, as found now in master). I gotta say, the stroke quality improvements really do seem to make a massive difference for making these strokes feel a lot nicer to draw with!

First up, introducing Mr Russ Prickles!

Russ' Briefcase - He works for Prickles Industries Ltd. - doing what, no-one knows...

I came up with the basic character design for these a few years ago while attending some particularly boring talks, while some of the characters/elements are ideas I've been toying with on and off for over a decade. They were originally going to be part of a little short film (or more specifically, the opening sequence of that) - with Russ hitchhiking his way home along a deserted, grass-lined moonlit highway, carrying his briefcase, back to his family in the hedgehog colony in the woods. Also, all of this was to have played out against the backdrop of Roger Whittaker's "Good Morning Starshine".

NZ Flag Doodle

Although I'm not in favour of changing the NZ flag, I nevertheless got inspired by some recent rants flying around to have a bit of a doodle for fun over lunch today. Here's what I came up with...

It turned out a bit like the "Red Peak" design in certain notable ways... it wasn't intentional, but that's just the way it ended up coming out organically... However, considering that I did come to see it as being a bit more agreeable than the other ugly disasters that the official committee ended up with, this quick hack also has some interesting properties I guess.

Grease Pencil - Lookdev Doodles

Just some after-work doodles with Grease Pencil, playing around with different colours and what effects can be achieved by playing around with different brush settings :)

As always, it was a great exercise for checking on the state of things. From personal experience, I can attest to how "dog fooding" (as it's often called in the software industry) is one of the best/most efficient ways for developers to pick up problems and/or workflow bottlenecks in their work!

Cosmos Laundromat - Episode 1 Online

Congratulations and thanks to the Gooseberry team for the belated (due to timezones) birthday gift this year :)

Mid Afternoon Sketches - Birdies

Some mid-afternoon doodles from earlier today - all with black ballpoint pen..

 Chiieee!

The silvereyes do sometimes get into a bit of a flap over a preferred perch...

A little warmup sketch

Quick Art Project - Drifting in Uncharted Waters

Last night, for the first time in ages I did some random doodling to relax a bit before heading to bed. Inspired by one of these, I spent a few hours tonight trying to convert these pencil sketches to 3D renders :)

.blend file