Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[dwsel]

Hello there,
I've been venturing into area of procedural (scripted) animation recently. This time it's the idea that I've mentioned some time ago - audio visualization:

The procedure to extract frequency bands from the music is quite simple:

1. Get Audacity http://audacity.sourceforge.net/download/ + LADSPA plugins pack http://audacity.sourceforge.net/download/plugins , Wavosaur http://www.wavosaur.com/download.php , Notepad++ http://notepad-plus-plus.org/download/v6.4.5.html
2. Open your audio (preferably mono downmix or a single channel)
3. Filter -> GLAME Bandpass Filter with parameters: a, a/10, 10
where a = frequency of the required band (for example 2000, 200, 10)
3. Filter -> Amplify - sound should be as loud as possible but the best the value should be equal for all channels/passes
4. Filter -> Diode Processor filter with value 1
5. Filter -> Waveshaper (acts like an expander making quiter sounds more quiet and louder more loud and beats more 'visible') - 1.2-2.2 should be ok
6. Filter -> Comb Filter with: Band separation = 100-300 and high-ish feedback = 0.95-0.99 - this is the 'smoothnes' or 'slowness' of the curve following the beats
7. Save the file and open in Wavosaur
8. Process -> Resample - enter Frequency equal to FPS of your animation (better not to use fractional framerates), with Apply interpolation = linear or spline
9. File -> Export -> Text
10. Open in Notepad++
11. Record a macro for a single line where you add '"$arr.push(' at the start of line, ');"' at the and and move to the next line and beginning (use keyboard only)
12. Repeat until the end of the file
13. Copy to appropriate location in the *.an8 file
14. Repeat for other channels/frequencies bands to create more audio lists (if you wish/that's required) - it's better to repeat exactly all values entered and change only the ones from step 3.

A hint: 'The best frequencies lie at exponentical curve. i.e I've used 125, 250, 500, 1k, 2k, 4k, 8k (7,2k to be accurate as the filter didn't allow me to enter 8k)'
Have a look here for such frequency lists: http://sound.westhost.com/project63.htm

Currently I'm working on tool to automate the process of injecting the processed audio lists into appropriate morphs' locations in *.an8 file, so that you could just create the *.an8 template and audio lists and using graphical tool inject them automatically without need for manual text processing.

[johnar]

 Thats really cool dwsel

Currently I'm working on tool to automate the process of injecting the processed audio lists into appropriate morphs' locations in *.an8 file

Even awesomer   

[ENSONIQ5]

I absolutely love this kinda thing, totally cool dwsel!  I agree with johnar, a graphical user interface to handle this would be seriously awesome.

[dwsel]

@ENSONIQ5, johnar: Thx! I'll see how it will look and work like.

@johnar: Now imagine if you'd be able to track individual phonemes sequence in the speech audio and plot a list of them to drive the mouth morphs Actually getting the phonemes from the audio is more difficult task than just plainly analysing given frequency ranges (you'd have to make some classification as well):
- https://ccrma.stanford.edu/CCRMA/Courses/152/speech_recognition.html
- http://basic-programming.blogspot.com/2005/10/speech-recognition-formants-for-vowels.html
- http://www.cs.dartmouth.edu/~dwagn/aiproj/speech.html
but it's another great idea for the future works.

[davdud101]

I'm not ASL user nor programmer or anything like, but this is COOL. Could you explain how it works more in layman's terms?


But DASSA LAAWNG RENDER TIME.

[dwsel]

I'm not ASL user nor programmer or anything like, but this is COOL. Could you explain how it works more in layman's terms?

I'm not very good at explaining things but let me try...

Have a look at this good demonstrational video:

=15s - although it's about circuit the method is near exactly the same used by me
- each 'peak' is called 'band' thus I'm using 'bandpass filter' for several times to extract each band from the whole audio during the process I mentioned
- the frequencies that are written under the each peak (and being heard during the presentation) are the ones used in his audio analyzer (note that I'm using the same ones except for the two lowest and one highest)
- the frequencies are 'a' parameter mentioned in step 3
- if you only need to extract beat or overall loudness of sound you can reduce the number of required steps significantly (even down to only 1. opening the file in Wavosaur, 2. applying a single lowpass/bandpass filter only (or even not), 3. downsampling to 24, 25 or 30 Hz = always equal to number of frames per second in your animation, 4. saving as txt)
- if you want to make your own simple audio reactive (of audio beats only for example) without multiple bands and all those bells and whistles just grab the Wavosaur http://www.wavosaur.com/download.php (application as simple, as light and as easy to use as Anim8or) and I'll guide you through the simplified version of process
- until I'll release tool to substitute keyframes/scripts in Anim8or files with the lists I can process your list to create text that you can easily paste in your morphs' script controller

But DASSA LAAWNG RENDER TIME.

Nay... not very long to me. You'd probably render the same thing on your own computer in a fraction of time (a couple of hours only maybe?) considering most PCs around me are not very high end Ooorrrr.. you can go for lower framerate like 15 FPS for example


I hope it sounds easier now and I'll encouraged you or someone else to experiment with this.