Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Steve]

With Build 1383 Anim8or has a new lip sync track to help add voice. More instructions to come in a few minutes.

Note: I updated the link to 1383 which fixes #101-036 - Anim8or crashes when reading a project that references non existent sounds.

How to use the Lip Sync track to a scene:

1. Start with an Object or Figure that has several Morph Targets that represent different speech phonemes (sounds like "AYE", "OOO", "MMM"). Give them a one letter name or the lip sync track will tend to get jumbled-up.
2. In the Scene editor, add one or more sounds. This creates a sound track.
3. Add the Object or Figure to the scene.
4. Use the Edit->EditPhonemeTrack menu command to add a lip-sync track to the EObject or EFigure. This will appear next to the sound wave in the sound window.
5. Double click on a frame in the lip-sync track to add a phoneme morph.

Existing morphs that conflict with the ones you add in the lip-sync track will be deleted, but you can still use them in other frames. All morphs that are not added to the track are not affected.

This example uses only 4 lip shapes, roughly for sounds A, O, M and N.

You can improve the quality with more shapes. For example, the very talented animator and teacher Preston Blair created a great scheme using 10 different shapes. Here is one description: http://www.garycmartin.com/mouth_shapes.html

I've attached a the project for this video. You can get the .WAV sound file here. When you load the project, switch to the scene editor, double click on the small blank square in sound track at frame 5 and load the sound file.

[captaindrewi]

1382 crashes when i try and load the example lip sync.an8

[Steve]

It looks like it crashes when trying to load the sound file. The directory that is in the project file doesn't exist. I'll see if I can get a fix out quickly.

[johnar]

i  made a test file with a ball and simple morphs in build 1382, and its looking good, and working as described
  Will use it more and get back with more feedback, but, so far so good.
 Nice one steve 

[Steve]

Update: Build 1383 should fix the crash reported by captaindrewi above. When you load the project, switch to the scene editor, double click on the small blank square in sound track at frame 5 and load the sound file.

[ianross]

So far so good

[dode1]

Perfect!

[johnar]

Maybe a zoom could work something like this?


 EDIT. I didn't put the phoneme track in there, but it should be.

[Nehemiah Thomas]

Hi! anim8or member congratulation for greater job. sorry there any way to solve the problem on anim8or build release 1383 because scene can't import figure

[Steve]

johnar Interesting idea, or maybe some other way to widen frames without limiting the active range. I'm working on some other ways to show longer names as well.

Nehemiah Thomas Yikes! This has been broken for a while. Importing scenes with figures can also crash. I'll get a fix out for this!

#101-039 - Import scenes with figures doesn't include figure; can crash.

[johnar]

 EDIT: Thanks for your reply steve

yes, i also had trouble import figure to scene..
 But i also have a bug to report. 

  'Anim8or forgetting edited phoneme keys'

 I used the phoneme track and set some morphs for the .wav file 'anim8or'
 These are just default, (0 and 1 value) morph keys, with no editing of morph value or positions.
 

 and these are the results. (remember .gif images play slower than film)
 


  Then i edited the keys as below. I used a variety of values, as seen in the graph editor, and moved the 2 keys, each side of the center key(s), away 1 frame from the center key(s).
 ie: from this: ---  to this: - - -


 and this is the result. (remember .gif images play slower than film)
 


  And heres the bug.
 after rendering the edited scene, i was left with this.
 
 The morph values had returned to 0 and 1, and extra 0 keys were added into the gaps which i had made.


In regards to the assigned default positions of the morph keys, when set with the phoneme track.
 Three keys in a row, side by side, with values of 0-1-0, will always cause a 'popping' effect, which is shown in the first 'mouth' .gif.
 To stop that, there needs to be, at least, one free frame between each key. As shown in the 2nd 'mouth' .gif.  (no popping).

 Is it possible to make that the default?
 so: - - -  instead of ---?

 Here's an afterthought, which would be really, really cool, and allow you to see at a glance where the 'value' key is.
 

 Often when tweaking phoneme morphs, the positions and number of keys can change. eg; when wanting to hold the mouth shut for 2 or 3 frames mmmore, or the sounds being longer than the normal 0-1-0. The coloured dots would show easily which is the initial 'value' key, which would make it much easier to keep track of the keys.
  (and it looks soooo cool). 

[Steve]

The phoneme track clears out any keys that are: 1) within +/- 1 frame something in the phoneme track, and 2) are used in the phoneme track. Then the keys form the track are added.

This poses several design issues:

A) Values other that 1.0 and 0.0 are needed.
B) Overlapping phonemes could be useful where one is transitioning into the other.
C) What is the best way to handle other, user added keys that conflict with phoneme keys?

Additionally, it would be nice to support:

D) Longer phoneme names.
E) Copy/paste in the phoneme track.
F) Clicking on the graph editor should edit phoneme keys accordingly.
G) Different colors will definitely help in the graph editor and the key markings.

I have work in progress for A and B and plan on addressing E shortly after.

Note: you can make phonemes longer than one frame in the dialog where you add a phoneme to the track.

[johnar]

Hi steve.
 Some thoughts on A, B and C.
 D, F and G are sounding good.
 E might be related to thoughts further below.

The phoneme track clears out any keys that are: 1) within +/- 1 frame something in the phoneme track, and 2) are used in the phoneme track. Then the keys form the track are added.
This poses several design issues:

 Is the phoneme track leaning towards being a little too 'involved' with the phoneme keys manually inserted by the user into the time track.?   please read further...

C) What is the best way to handle other, user added keys that conflict with phoneme keys?

 I think they need to behave 'as one' as much as possible, but with absolute precedence given to 'user defined' keys. Don't allow 'Lip sync tool' added morphs/keys to change any already existing keys. (except, maybe, to overwrite if it lands on an existing key)


 Lip sync tool is excellent to 'mark' which shape goes where and when.
 Anim8or lip sync tool is doing that now, which is a huge help.
 
 If it can actually add the key(s) as well, then yes, that is very good, but guaranteed it will need tweaking keys, (hooray graph editor), and 'moving keys', here and there. to get it right.
 
 I don't believe there is any automatic lipsync software that can get it looking as natural as keying manually.
 For a 'convincing, life like' lip sync, you will always need to be able to edit keys. in the timetrack, and graph editor.

 

 I would like to talk about popping again, sorry. but its relevant to:
 

A) Values other that 1.0 and 0.0 are needed.

 So, when keys are added to the timeframe, using the lipsync tool,are they not added as just ordinary keys? 'off(0)-on(1)-off(0)'
 To avoid popping, there will need to be 1 frame between the first 'off(0)' and the 'on(1)'
 If this frame is left empty, won't it just travel at a natural strength/curve value, as do ordinary  strength values change between 1 key frame value and another?

Note: you can make phonemes longer than one frame in the dialog where you add a phoneme to the track

 Yes, that holds the morph at 1, but again, there needs to be an empty frame at the end, between the 1 value and the 0 value, if you want to eliminate the popping.

 With the lipsync tool, I honestly believe that if you can add a default keyframe pattern. of:
 '0 - empty frame - 1 - (held if lengthened in the dialog) -1- empty frame - 0',
 then it will work better than any lip sync apps i've seen.

 The keys should be added as ordinary keys, and not even necessarilly 'physically' connected to the Letter in the phoneme track, once the phoneme, (and 3 keys), have been added.
 (maybe the main value key can still have a relationship with the phoneme track letter.  but only to the point of deleting, if the key is deleted from the track. (and vice versa?),
 
 

B) Overlapping phonemes could be useful where one is transitioning into the other.

Absolutley. And without any real physical connection between phoneme track and phoneme key, after the 3 keys are added, could that not then allow overlapping transitions, because they're just more keys on a seperate track,  editable and readable as any other overlapping morph keys?


 

[ianross]

 Firstly thank you Steve for the new sound builds, the second sound build
 proved to be very interesting. My test character had 4 phoneme mouth morph
targets  and I recorded my voice with audacity free software, I then added
the sound track. Then from the drop down edit menu, I selected  edit phoneme track and I selected my test character head. I then played my
sound track in the scene editor and paused it at the time frame I wanted the
mouth to move. At this point I clicked in the phoneme track and selected the
mouth shape to match the sound, in this case it an “O” sound. See picture 1.
Anim8or produces a default value of 1 (which you can change) it also develops a default value of 0 on the key frames before and after your selected frame. This is very good and helps to save time, see picture 2.
 Can we play the sound track at a slower speed? It was difficult to select the exact area of the sound wave because it played at normal speed, which proved to be very tricky.

[ianross]

Steve has done  wonderful work with the creation of the phoneme track. Please review the papagayo you-tube tutorial below.
At 3.17 minutes into the video he scrubs along the soundtrack and you can hear the broken down words and phonetic sounds.
Can this scrub/scrolling of the wave track be included into anim8or?