Anim8or Community

General Category => Anim8or v0.98 Discussion Forum => Topic started by: spicy on March 28, 2008, 04:32:48 pm

Title: Lip Sync
Post by: spicy on March 28, 2008, 04:32:48 pm
 :) Hello everyone I was just thinking. It would be really good if Anim8or had some lip sync. Instead of morphing the targets. It would make life alot easier ;)
Title: Re: Lip Sync
Post by: Pirate92 on March 29, 2008, 11:06:06 pm
What do you mean?
Title: Re: Lip Sync
Post by: RnDr FOX on March 30, 2008, 12:22:48 am
Yea I know what you mean and that would be awsome, but I dont know if steve would be able to do that.
Title: Re: Lip Sync
Post by: hihosilver on March 30, 2008, 08:12:58 am
Yes, what do you mean spicy?  I don't have any idea of what you're talking about.  Lip syncing is not a part of a program, it's a way to imitate the positions of a mouth when speaking.
Title: Re: Lip Sync
Post by: spicy on March 30, 2008, 02:40:38 pm
Yes I know that Hihosilver. What I was suggesting was that, it would be good if steve could import some type of lip sync into Anim8or. Then it would be alot easier to move the lips when talking (In animations)
Title: Re: Lip Sync
Post by: hihosilver on March 30, 2008, 08:36:24 pm
Do you mean to import sound files?
Title: Re: Lip Sync
Post by: spicy on March 31, 2008, 05:22:25 pm
no. i mean that it would be good if anim8or could have it's own lip sync program that we could use to more easily move the lips during animation
Title: Re: Lip Sync
Post by: hihosilver on March 31, 2008, 05:40:57 pm
Could you explain how that would work?  I'm sorry for all the questions, but I'm trying to make this more clear, as what you're saying is very general, and it would probably be nice to know how you would like things improved.
for one, I'd like to be able to have a bar you could drag or arrows you could hold that would change the morph value and update in realtime.
Title: Re: Lip Sync
Post by: thecolclough on March 31, 2008, 09:33:56 pm
i'm not sure exactly what spicy was thinking, but he's given me an idea here - how about the ability to play back another video within anim8or's scene mode, with the playback set up in such a way that you can play, or step forwards or backwards, in both the anim8or scene and the other video simultaneously?  i.e. you advance one frame in the anim8or timeline, and the video also advances one frame, etc etc.  that could be helpful for loads of stuff, because you'd then have an easy way to match your animation (e.g. lip movements, or even some types of physics simulation) to a live-action reference.  from what i've seen on behind-the-scenes documentaries, it looks like the pros (e.g. at pixar) do that sort of thing all the time.

- colclough
Title: Re: Lip Sync
Post by: xalener on April 03, 2008, 12:38:11 am
Rotoscoping/matchmoving is what you're thinking of...
Title: Re: Lip Sync
Post by: thecolclough on April 03, 2008, 08:13:29 am
Rotoscoping/matchmoving is what you're thinking of...

i guess.  i thinking in terms of just having the reference video in a separate little window (like the graph editor one), rather than animating over the top of the video (i.e. rotoscoping per se), but now you mention it, that'd be cool too :)

- colclough
Title: Re: Lip Sync
Post by: lynn22 on April 03, 2008, 08:52:14 am
..... what i've seen on behind-the-scenes documentaries, it looks like the pros do that sort of thing all the time.

- colclough
I guess we would all like to have the equipment and software we see the pros use but are we also willing and, more so, able to dish out the money those pros did for that kind of equipment and software ? ;)
Title: Re: Lip Sync
Post by: Tanzim on April 03, 2008, 09:14:20 am
That would be cool but Steve had said just don't ask for things that other programs have
Title: Re: Lip Sync
Post by: thecolclough on April 03, 2008, 09:16:57 am
I guess we would all like to have the equipment and software we see the pros use but are we also willing and, more so, able to dish out the money those pros did for that kind of equipment and software ? ;)

of course not, that's why we keep asking steve for new features in anim8or instead.  if we could afford pro stuff, we probably wouldn't be on this forum - but the anim8or way is so much more economical, and therefore much, much better ;D

- colclough
Title: Re: Lip Sync
Post by: SirBonovox on April 03, 2008, 12:29:12 pm
Hi guys, just wanted to throw my two cents in.  I agree with Hihosilver, a slidebar would be great for manipulating things like morph targets, but until then the graph editor works pretty good.  The only way I can see Anim8or having builtin functions for lipsyncing is if it had a standard mouth that everyone had to use for their models.  It's hard to come up with a way to add shapes to a mouth to form words and somehow apply that to every mouth anyone can make.  UNLESS...(this is where I get to the cool part  ;D )...we use motion capture!!  Something thecolclough said gave me an idea.  First of all, as far as the seperate video window you would use for motion matching, can't you do that very same thing with something like Windows Media Player or some kind of video editor?  You could even put dots on someone's lips, record video of them talking, play that video frame-by-frame in Windows Movie Maker and match-up the dots to corrosponding points on your model.

Here's another thing I've been thinking of.  Ever since I heared of that Icarus program I thougt there must be some way to do motion capture in Anim8or.  I haven't used it yet (so I may not know what I'm talking about) but according to the video tutorial it can be used to create 3D points of refference in Blender, and from what I understand, it can do the same for Anim8or you just have to edit the .an8 file.  It seems to me that if the program can track the background movement it should be able to track any kind of movement, say little white balls stuck all over someone. ;)  The location of these balls gets put into the .an8 file and somehow is used to move corrosponding points on a model and "IT'S ALIVE....IT'S ALIVE...HAHAHAHA" 

Maybe I'm just dreaming but I wanted to throw it out there.
Title: Re: Lip Sync
Post by: SirBonovox on April 03, 2008, 12:41:48 pm
OK, so I've rethought my idea and realized that Icarus is used to track movement on a 2D plane, not 3D space.  Again, this is just as far as I understand, but I could be wrong.  But still, it may work for lipsync :-\
Title: Re: Lip Sync
Post by: thecolclough on April 03, 2008, 03:34:52 pm
*snip*  as far as the seperate video window you would use for motion matching, can't you do that very same thing with something like Windows Media Player or some kind of video editor?  *snip*

you can, that's what i did to match up the mouth movements in youtube.com/watch?v=i9y85zRtkgg[/youtube]]this movie (http://uk.[youtube) (yes, i know it's a bit rough, but that's basically because i was lazy in building the mouths, and the youtube server has desynched all the audio anyway...  :-\ ); i was just thinking it'd be cool if anim8or had its own built-in video playback window, so that the reference video and the animated scene could be tied together, and moving forwards or backwards any number of frames in one, would also move the other, so you didn't have to keep worrying about whether or not you'd moved them the same number of frames each.

i've never experimented with putting reference marks on faces when doing it (partly cos it was my own face, and i didn't think black dots around the mouth would really suit me  :P ), but yes, that's entirely feasible - if your cast are willing to do it!   ;D

- colclough
Title: Re: Lip Sync
Post by: SirBonovox on April 03, 2008, 06:23:12 pm
That's actually what they did for the Beowulf CGI movie.  They did full body motion capture on all the actors and even drew lines and put white dots on their faces.  It looked pretty funny but it worked. :o ;D
Title: Re: Lip Sync
Post by: almost cool on April 03, 2008, 11:55:31 pm
how about you set the directional motiont of the mouth, and then determined by the sounds frequency and power, have it move in sync with the sound! 8)