Anim8or Community

Please login or register.

Login with username, password and session length
Advanced search  

News:

An update to Anim8or, v1.00b, is available with a few bug fixes. Get your copy HERE. See the "ReadMe" file for details.

Author Topic: lipsync and animation 2  (Read 644 times)

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
lipsync and animation 2
« on: June 15, 2020, 04:08:42 am »

Hi again,

 I've spent the last wee while, torn on where i should post this. Here, or in the Lipsync tool thread.

 Because of the current work being done on an8 lipsync tool, and because of the pics and short vid which i've made with 'anim8or lip sync' in mind, i've decided to post it here, rather than clog up the lipsync tool thread with, essentially, only my way of doing lip syncing
 But by posting here, i'm also hoping it may still be helpfull, somehow, in the building of the an8 lip sync tool, and the syncronization of it into a simple workflow of lip syncing in Anim8or.

 First off, here's the example video. (it repeats to show mouth shapes)


 So, step 1 is identifying which phoneme goes on which frame of the timetrack.

 This is probably the hardest and most important thing i've found with lip syncing, so steve, theres plenty of crucial work there, and am looking forward to that journey with you.
 
Up until 'sound and lipsync' coming to anim8or, i've used a trial version of 'magpie pro'. The full version has plenty of bells and whistles, but all u need is a picure of the .wav file with corresponding frame numbers, Magpie trial has a 'select and zoom' function, and 'arrow key scrubbing'. These help tremendously in locating the right frame with the right sound.
 I'm old school, so i record the frames like this:


 With this info we're good to go.
 
 You'll notice by the numbers in the pic above, that normal speed speech doesn't have a lot of spare frames in a word. In fact, there's often only 2 frames between 1 phoneme and the next. This makes  'Overlapping phonemes where one is transitioning into the other' quite a critical part of getting realistic lip syncl.

 Heres the time track with keys set for "family of thieves"

 The keying itself is actually quite simple, the trick to realism is being able to use the graph editor to adjust the strength of the keys.
 Another important trick is to allow a mimimum of 1 empty frame between each phoneme key, to avoid 'popping', which really doesn't look good, and wants to be avoided if you want your lipsyncing to look ok.

 And heres a gif of the results (because i love making animated .gifs)

 "Family of Thieves"
 I'll contiue posting progress of this new 11 second clip as i get through it, and welcome any feedback
 Ideas of anim8ors new sound and lipsyncing should be placed in the appropriate threads please.

 Thanks.   

 Audio courtesy of 11secondclub.com   Thank you


« Last Edit: June 15, 2020, 08:16:41 am by johnar »
Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: lipsync and animation 2
« Reply #1 on: June 15, 2020, 06:26:32 pm »

Spectacular! And you have some good advice for lip-syncing, johnar.
Logged

dode1

  • Newbie
  • *
  • Posts: 12
    • View Profile
Re: lipsync and animation 2
« Reply #2 on: June 15, 2020, 06:37:12 pm »

I am probably stupid, but is there sound with these animated gifs? I like gifs too, but are we incorporating sound with these morphs or what? I saw the video with sound and that is great! I made my first morphs today following helpful posts here and on Youtube.
« Last Edit: June 15, 2020, 06:40:25 pm by dode1 »
Logged

thecolclough

  • Sr. Member
  • ****
  • Posts: 866
  • 3D Ignoramus 1988-2002, Anim8or User 2002-present.
    • View Profile
    • www.mattghc.com
Re: lipsync and animation 2
« Reply #3 on: June 15, 2020, 06:47:17 pm »

lots of good work there, johnar.  i completely agree that phoneme identification is hard work, and i must admit i haven't made a serious attempt at it myself in years :P ...although i might have another stab at it now that steve's given us a shiny new tool for it!

if i might offer a spot of constructive criticism: one thing i've noticed from watching your character on a loop is that his top teeth are moving relative to his skull.  makes sense for the bottom teeth to do that, of course, but not so much the top ones, so it might be worth disconnecting those from the flexible mouth element and rigging them separately so they stay static.

dode1, no, the .gif format doesn't support audio.
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: lipsync and animation 2
« Reply #4 on: June 15, 2020, 09:07:51 pm »

Thank you steve. Thank you heaps. That means a lot to me.


thecolclough
Quote
if i might offer a spot of constructive criticism

 Yes Please!!
  Well spotted with the top teeth. I thought i was getting away with that.
 
 A while back, his top teeth were annoying me in some particular mouth shapes, ('FFF' was the main one) so i added a morph target to lift or lower them, just to tweak things when polishing in scene mode. I did the same with the bottom teeth, you can see the morphs in the left side of the pic of the time track. ('topteethup' and 'bottomteethup' ) ('down' is just a negative value of up.)

 But now that you've pulled me up on those top teeth, you're absolutely right, and 'hopefully' you'll notice, in the future, that i've taken your advice on board.

 thanks.   

dode1 keep at it with those morph targets. Once you get comfortable with them, it's amazing the 'scope' of what can be done with them.
« Last Edit: June 15, 2020, 09:08:34 pm by johnar »
Logged
%

ENSONIQ5

  • Sr. Member
  • ****
  • Posts: 965
    • View Profile
    • Mission Backup Earth
Re: lipsync and animation 2
« Reply #5 on: June 16, 2020, 04:39:53 am »

Love your work johnar, and great to see I'm not the only one who regularly falls back on good old pen and paper!  I admit that I am yet to tackle lip syncing, it's one of those "one of these days" things, but I know who to hit up for advice when I do ;)
Logged

ianross

  • Full Member
  • ***
  • Posts: 179
  • Imagination is the elixir of life.
    • View Profile
    • https://www.tradebit.com/filedetail.php/164976574-part-1-anim8or-tutor-course
Re: lipsync and animation 2
« Reply #6 on: June 17, 2020, 07:12:47 am »

Johnar this is a very good post, thank you. Also, Papagayo is a lip-syncing free program designed to help you line up  phonemes (mouth shapes) with the actual
recorded sound of actors speaking. Papagayo makes it easy to lip sync animated characters by making the process very simple.
This software is very useful creating and plotting out the voice overs for your anim8or
movies.You will need to develop storyboards in order to get successful    synchronizations.

 
         
         
           
         
Logged

Kevin Gales

  • Jr. Member
  • **
  • Posts: 90
    • View Profile
Re: lipsync and animation 2
« Reply #7 on: June 17, 2020, 03:59:03 pm »

Ianross gets it...I personally use papagayo and it's pretty cool...If you check my lip sync posts...I used papagayo there...
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: lipsync and animation 2
« Reply #8 on: June 18, 2020, 08:36:11 am »

ENSONIQ5
 Long time no see.
 Yes, don't know where i'd be without the good old pen and paper.
 Thanks for the post. Hope all is going well in your 'digital endeavours'
 (havn't seen anything from you in a while. )

thecolclough
 I'll do something with the 'top teeth' setup before/for the next one.

steve
Doya like the glasses?
Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: lipsync and animation 2
« Reply #9 on: June 18, 2020, 03:53:20 pm »

The shades are cool! (and the lip-sync ain't bad, either!)
Logged

ENSONIQ5

  • Sr. Member
  • ****
  • Posts: 965
    • View Profile
    • Mission Backup Earth
Re: lipsync and animation 2
« Reply #10 on: June 19, 2020, 06:51:31 am »

ENSONIQ5
 Long time no see.
 Yes, don't know where i'd be without the good old pen and paper.
 Thanks for the post. Hope all is going well in your 'digital endeavours'
 (havn't seen anything from you in a while. )

Yes, it has been a while since I've posted anything, I'll pop something up soon for old times' sake :)
I log in regularly to check out updates, see what people are up to and generally lurk a bit.  Still working on the Mission Backup Earth project, mainly live action over CG backgrounds so no lip-sync required yet ;)
Logged

davdud101

  • Hero Member
  • *****
  • Posts: 1031
  • Couldn't ever use Blender or Maya. Anim8or's good.
    • View Profile
Re: lipsync and animation 2
« Reply #11 on: June 19, 2020, 12:03:41 pm »

That's some real good stuff johnar!

Gotta say Steve, great implementation, this couldn't have come at a better time (right when I'm messing around with lipsync on a character). Excited to see what else the community does with it!

@ENSONIQ5 Been a while!!
Logged
?