Anim8or Community

Please login or register.

Login with username, password and session length
Advanced search  

News:

An update to Anim8or, v1.00b, is available with a few bug fixes. Get your copy HERE. See the "ReadMe" file for details.

Pages: 1 2 [3] 4

Author Topic: Lip Sync Tool  (Read 3096 times)

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: Lip Sync Tool
« Reply #30 on: July 05, 2020, 11:43:35 am »

Regarding sampling rates, there are tons of free sound effects and such available, but they have different rates. It'd be easier to use then if you didn't have to do the conversions. Besides, learning a bit of new math ain't all that bad
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #31 on: July 05, 2020, 07:00:41 pm »


Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: Lip Sync Tool
« Reply #32 on: July 07, 2020, 06:52:41 pm »

johnal I've run into another issue with copy/pasting phoneme blue keys. They aren't independent. Each phoneme starts and ends with a 0 key on either side on the key.

So what does it mean to copy a phoneme key? What I propose is:

1. Copying only a zero key doesn't really mean anything. Ignore it when this happens.
2. Copying only one non-zero key is pasted as a one frame phoneme, with a default 2 frame lead and 2 frame tail if those aren't also selected.
3. Copying all or parts of multiple keys treats each key independently according to these rules.

I realize that this is a bit confusing but I haven't been able to come up with a better solution. Any ideas?

BTW I have scrubbing working for the arrow keys and scrubbing. It plays single frames so kind of stutters. I'm trying different ways the make the scrubbing sound smoother. (But scrubbing backwards will always stutter since I cant play the sound in reverse.)
« Last Edit: July 07, 2020, 06:53:36 pm by Steve »
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #33 on: July 07, 2020, 10:46:00 pm »

 Steve.
 Am thinking of options re copying phoneme keys. Will get back to you on that in a day.

 re scrubbing....awesome.
 re stuttering, I tried to upload a zip with a video in it of play option stuff in magpie pro just for interest and comparrison sake. Unfortunately too large for attachment so have emailed it to you.

EDIT: Sorry i got rushed there steve. After checking the video, the sound is pretty bad,
 I shouldve made it a bit shorter to cut down on size and audio quality.
  Although the sound quality is bad, i just wanted to give you an example of what it sounds like frame by frame in other audio/lip sync software, as maybe a sort of comparrison thing.
 It does sound sort've 'stuttery' anyway, being the nature of 1 frame of sound.
 But thats probably not the 'stuttering' you were meaning anyway.
« Last Edit: July 08, 2020, 12:26:56 am by johnar »
Logged
%

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #34 on: July 08, 2020, 08:24:12 am »

Quote
1. Copying only a zero key doesn't really mean anything. Ignore it when this happens.
2. Copying only one non-zero key is pasted as a one frame phoneme, with a default 2 frame lead and 2 frame tail if those aren't also selected.
3. Copying all or parts of multiple keys treats each key independently according to these rules.

I realize that this is a bit confusing but I haven't been able to come up with a better solution. Any ideas?

Yes i think i see the problem, as explained in number 3.

 Keeping in mind that a copied non-zero key is pasted with a default lead and tail key, then it makes sense that this scenario will always cause issues if overlapping occurs.
 The problem has potential to become 'overwhelmingly confusing' when "Copying all or parts of multiple keys"
Two possible solutions.
 
1) So far, the easiest solution i can see to this is to not allow overlapping.

 Then, how would this work, when applied to "Copying all or parts of multiple keys"
 If existing keys are 'deleted if overlapped', including any 0 keys, would that then 'clear the way' for the 'non-zero and its default lead and tail keys'?

  Keep in mind that, in my experience at least, when animating lipsync manually, the chances of ever 'needing' to 'overlap the same phoneme key' in the timetrack is .........never (?)........

 As mentioned earlier, a warning to the user that 'keys will overlap and affected existing keys will be deleted', would work.

 So, to clarify, one possible solution that i can see at this point, is to:
 'Not allow overlapping, and if overlapping is going to occur, delete any existing keys from the timetrack that will be overlapped'

 Would that work?

EDit:
 Possible solution 2} If copying a 'block' of phoneme keys, to repeat further along the timetrack, causes problems when 'pasting'  due to phoneme key 'rules' then would it be possible to somehow change those phoneme keys to ordinary keys, as is done when overlapping/adding keys into a sequence, in scene mode? Then the keys to copy would just be ordinary keys, and the phoneme key rules would not apply
« Last Edit: July 08, 2020, 08:44:20 am by johnar »
Logged
%

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #35 on: July 09, 2020, 09:18:11 am »

Steve
Still thinking about copying phoneme keys.
 Possible solution number 3, and i think this one might be my favourite so far.

I guess the thing is, do you want the user to:
 1) 'use' the timetrack, along with the the phoneme key track/editor, or
 2) be able to create credible lipsync using only the phoneme key track/editor.
 (and graph editor in both cases)

 My vote would go for 1. (tho i think this possible solution caters for both 1 and 2.)
 The 'timetrack and keying' setup in Anim8or is excellent, and simple to use. Animation requires a knowledge of keying, so it's a good thing to learn a little bit about using keyframes. (even if you only want to do lipsyncing).

 With that in mind, maybe the phoneme keys need to be a bit more flexible somehow.
 What if: Set a Ph.Key by double clicking on the track. Blue phoneme keys can be edited using the PhKey editor and graph editor. (as it is now)
And then, later,  you can double click the name in the phkey track, and change values and settings in the editor, to change the blue keys in the track. (as you can now)

 Now, what if, any phoneme key manipulations done in the timetrack, or graph editor Cancelled all connection with the phoneme key editor. Those keys, (lead, Value and Tail), would lose their blue status, and the name in the phkey track could turn red, to show its disconnected from any keys.
(Double clicking on a red name in the phoneme track would say that there are no keys connected in the timetrack, and an option to leave or 'delete' the phoneme name)

  Once keys are disconnected from the phoneme key editor, they'll be normal keys, and will be free to copy, cut and paste until the cows come home.

 So, Initial phoneme keys can still be set, and edited, through the phoneme key track and editor,  and would be capable of producing credible lipsync using only them.
  More advanced tweaking and manipulation initiated through the time track, including being cut or copied, is still possible as well, if required.
 Could be the best of both worlds, and really pretty straight forward.

 re the graph editor. If the phoneme key editor is open, the blue key values can be edited in the graph editor. (as it is now)
 But if the phkey editor is not open, and the key has been selected through the timetrack or graph editor independently from the phoneme key editor, then that would warrant 'disconnection'.
 Does any of this have potential?

 


« Last Edit: July 09, 2020, 09:49:37 am by johnar »
Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: Lip Sync Tool
« Reply #36 on: July 09, 2020, 06:32:27 pm »

johnar: give build 1386 a try. I added sound when scrubbing and it seems fairly good.

I also fixed a very old issue: the different subdivision levels for working and final views never worked. So for subdivision objects you can set a low number, such as 2, for editing and a higher one, such as 3 or 4, for the rendered images. This can really help the responsiveness with sound and larger models.
« Last Edit: July 09, 2020, 06:32:47 pm by Steve »
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #37 on: July 10, 2020, 01:36:27 am »

 1386 Theres no phonemes showing in the morph section of the phoneme editor.
 Oops. must've forgot to reconnect that wire.  ;)

 Excellent news that the scrubbing is working out. That's awesome. Looking fwd to checking it out.
  And Well done fixing the old subdivision levels, therefore helping responsiveness with sound, (and larger models).
 Good thinking.
« Last Edit: July 10, 2020, 06:52:02 am by johnar »
Logged
%

Nehemiah Thomas

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: Lip Sync Tool
« Reply #38 on: July 10, 2020, 09:00:14 am »

Hi, congratulation for build 1386 there's some problems
1. subdivision skinning object not working at sequence and scene
2. crash on image and video rendering

Good working team for anim8or build 1386
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: Lip Sync Tool
« Reply #39 on: July 10, 2020, 09:40:29 am »

Skinning is broken? Rats! I know I'd break something. It was a lot of changes. I'll get right on it.
Logged

Kevin Gales

  • Jr. Member
  • **
  • Posts: 90
    • View Profile
Re: Lip Sync Tool
« Reply #40 on: July 11, 2020, 12:39:05 pm »

Scrubbing...pretty sweet 😁
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #41 on: July 11, 2020, 06:59:13 pm »

Hi steve

 Seems like theres no phonemes showing in the morph section of the phoneme editor for figures

 Its working for objects, so i got to try the scrubbing.
 Awesome. Actually clearer than the software i've been using, and super responsive.
 Really awesome.

EDIT: woohoo.

 Now that i'm getting some time in, i'm seeing how much work you've done, and how well it's coming along.
 I'm blown away steve, you're a champion!.
« Last Edit: July 12, 2020, 02:48:35 am by johnar »
Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: Lip Sync Tool
« Reply #42 on: July 12, 2020, 01:51:26 pm »

johnar, I can add a phoneme track and phonemes to figures. Do you have a simple example that shows the problem? (There is a little bug that when you delete a figure's phoneme the two blue zero keys aren't deleted.)

I almost have the skinning bug fixed. Too many cases!!

(And thanks for the nice comments )
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1891
    • View Profile
Re: Lip Sync Tool
« Reply #43 on: July 12, 2020, 08:30:33 pm »

 OK build 1387 should fix the skinning problem :)
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 928
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #44 on: July 12, 2020, 10:24:54 pm »

 Some good news steve,
 small misunderstanding.
 By phoneme not showing for figures, i was meaning:
In the phoneme  editor:

 When clicking on arrow to show morphs, theres nothing showing. (figure is loaded)
Like this:


 But, good news is, its fixed in 1387!
 So all is good.
 Will post back later, i,ve got some more feedback etc, but not at home right now.
 Just wanted to clarify the 'phoneme and figures' thing.
 All good now. (better than good actually)
 Thanks.
« Last Edit: July 12, 2020, 10:27:30 pm by johnar »
Logged
%
Pages: 1 2 [3] 4