Anim8or Community

Please login or register.

Login with username, password and session length
Advanced search  

News:

An update to Anim8or, v1.00b, is available with a few bug fixes. Get your copy HERE. See the "ReadMe" file for details.

Pages: 1 [2] 3

Author Topic: Lip Sync Tool  (Read 1090 times)

ianross

  • Full Member
  • ***
  • Posts: 172
  • Imagination is the elixir of life.
    • View Profile
    • https://www.tradebit.com/filedetail.php/164976574-part-1-anim8or-tutor-course
Re: Lip Sync Tool
« Reply #15 on: June 18, 2020, 05:55:49 am »

Also I am having difficulties with my "img" coding, they are not showing up in the forum, my pictures where originally  posted to this website imgbb.com, then I use this at the start of the address  .
The result is this and it does not show up in the anim80r forum img]https://ibb.co/VH0QbcZ[/img]
Can anybody help? ???
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1852
    • View Profile
Re: Lip Sync Tool
« Reply #16 on: June 18, 2020, 03:51:38 pm »

I think you have to have an image extension on the URL: .gif .jpg .png
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1852
    • View Profile
Re: Lip Sync Tool
« Reply #17 on: June 19, 2020, 03:02:44 pm »

ianross Papagayo looks pretty nice. It shouldn't be too hard to incorporate their phoneme output into Anim8or.
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 901
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #18 on: June 20, 2020, 12:59:32 am »

Is this the same Papagayo first posted here by nero, maybe 4-5 years ago? (possibly more like 10 years ago)

( I have a feeling he may have done the original programming.............(?))


ianross i notice on your image coding example that you're also missing the '[' at the beginning of ' img]https://ibb.co/VH0QbcZ[/img]
« Last Edit: June 20, 2020, 08:31:00 am by johnar »
Logged
%

ianross

  • Full Member
  • ***
  • Posts: 172
  • Imagination is the elixir of life.
    • View Profile
    • https://www.tradebit.com/filedetail.php/164976574-part-1-anim8or-tutor-course
Re: Lip Sync Tool
« Reply #19 on: June 21, 2020, 10:14:46 am »

Hi Steve, can the ".dat"  export file from papagayo, can a script be written for anim8or to
interpret this for morph targets? would it work in the phoneme track?
The dat file can be read in note pad.
Logged

ianross

  • Full Member
  • ***
  • Posts: 172
  • Imagination is the elixir of life.
    • View Profile
    • https://www.tradebit.com/filedetail.php/164976574-part-1-anim8or-tutor-course
Re: Lip Sync Tool
« Reply #20 on: June 21, 2020, 10:18:31 am »


Hi Johnar what file sharing website do you use? The brackets are in place>
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1852
    • View Profile
Re: Lip Sync Tool
« Reply #21 on: June 21, 2020, 11:32:14 am »

ianross: I 'm going to add built in import for papagayo .dat files.
Logged

Kevin Gales

  • Jr. Member
  • **
  • Posts: 63
    • View Profile
Re: Lip Sync Tool
« Reply #22 on: June 21, 2020, 03:54:40 pm »

ianross: I 'm going to add built in import for papagayo .dat files.
Yeah do that..the dat file contains plain text and pretty straightfoward
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 901
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #23 on: June 21, 2020, 11:49:51 pm »

ianross Hi.
 I use postimage.org. They're really good.
 just copy the 'direct link' they give you, paste here, select, and then click the little 'insert image'  button, below the 'B', above the 'reply to topic' box.

It puts  [/img]   at the end of the link  and  [img]    at the begining. 
 I  noticed your first '[' was missing in your example above.
 and yes, the link provided by postimage has  a file extension.  (.jpg)
« Last Edit: June 21, 2020, 11:58:39 pm by johnar »
Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1852
    • View Profile
Re: Lip Sync Tool
« Reply #24 on: June 29, 2020, 07:16:19 pm »

johnar, try out Build 1385. You can change the number of frames leading into and trailing a morph so that they don't pop, and can set the strengths of the start and end. I also marked the phoneme keys in color (nice suggestion!).

Also when you modify the graph editor keys the phoneme references are updated accordingly( instead of getting lost when you saved the file).
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 901
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #25 on: June 30, 2020, 09:20:32 am »

Hi steve.
 Just a quick note to let you know i've downloaded 1385 and have only had time for a quick look so far. I'll have some more feedback within the next couple of days, but for now, all i can really say is ---FAN TASTIC----
 A great variety of options now in the Phoneme Editor, have tried just a few different mixtures of settings, and so far all is looking really good.
 A great amount of control for setting initial phoneme keys. Gotta say that's quite outstanding. Something quite special. Well done.
 Sorry i don't have time for a closer look right now, but can't wait to take it for a longer spin. Exciting stuff steve, looking really, really good.

 One thing.
 When deleting a phoneme key, the first and last/outside keys aren't deleting.
 (the '0' keys).
 When you save the file, close it and re-open it, then the outside keys have gone.
 And
Quote
modified key values are saved
.

 Gotta go for now.
 Magic stuff steve
.
Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1852
    • View Profile
Re: Lip Sync Tool
« Reply #26 on: June 30, 2020, 10:22:00 am »

Thanks, johnar! I'll fix the undeleted keys in the next build.
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 901
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #27 on: July 03, 2020, 05:12:16 am »

Hi again steve.
 I'm not finding any problems here, except for that keys not deleting thing.
(thanks for the coming fix)

One thing i should mention.
 A tip for anyone using the phoneme track, which is probably best as a 'rule' for editing phoneme keys. (Unless this is something that you decide is worth fixing. But it's not broken, so this 'rule/tip' is probably all thats really needed)
 I'll try to explain.
 I set a phoneme key and have the 'fade' period set to (eg) 4. Later, when 'fine tuning/tweaking the keys. that key needs to be moved to (eg) 5. (or 6 or 2 or whatever).

 The rule would be, "when editing phoneme keys, changing the length, lead, or fade value should be done within the phoneme key editor".
 Why?
 Initially, i wanted to extend the fade key value from 4 to 6. or 'move that key 2 frames further along the track. So i just Ctrl-X that key, and paste it 2 frames further along the time track.
 It works, of course, but you lose the blue color from the key. After a bit of similar editing, the time track is left with an 'apparently' random smattering of blue keys, Which sort've defeats the purpose of coloring the phoneme keys in the first place.
 Maybe not necessarily a rule, but at least a tip.
 (unless you think its worth looking into)
.

 Apart from that, i think what we've got right now is absolutely cool, functional and practical. Capable of far better results than can be expected from any 'automatic lip-sync app or function'.
 Having this 'hands on' ability to set the keys 'placement and values' easilly and at the first instance, also makes things a whole lot easier when fine tuning later on.
 I have experimented with 'automatic lip-sync software and I'm totally confident that anim8or has the beginning of something far superior than anything i've seen.
 I say 'the begining of something', because, as we know, the hardest part of lipsyncing is putting the phoneme in the right frame at the right time.
 Scrubbing is probably what most people look for, and then some kind of 'zooming in' or 'magnifying', to have a closer look at where we think the sound is actually made.
 As we are now, we can select an area of the waveform and play it. That actually does work quite well and, with practice, the timing can be set very closely, if not spot on. (some waveforms are easier to read than some others).

 I've mentioned 'arrowkey scrubbing' If scrubbing is used for finetuning of phoneme placement, then i would say that 'arrow key scrubbing' is for even finer tuning. (more exact placing). If you can advance 1, or 2, frames at a time, with sound 'on', then synchronizing can be 'exact', and you can't get any closer than that. 

 Think i've done my rave for now.
 Great job steve.
 Cuppa tea

EDIT sorry for my, often, long winded, roundabout way of saying things sometimes.
 I realise now that perhaps what i could have said was 'Is there any way the phoneme keys can retain their color after being cut/copied and pasted'?
« Last Edit: Yesterday at 05:15:02 am by johnar »
Logged
%

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1852
    • View Profile
Re: Lip Sync Tool
« Reply #28 on: July 03, 2020, 01:02:10 pm »

Thanks for pointing out the copy-pasting keys behavior. I'll see if I can make this alter the phoneme properly.

I'm also thinking about allowing the phonemes to be modified directly in the phoneme track. It takes a bit of experimenting to see what works smoothly.

Then there are issues with overlapping lead/tails of two adjacent instances of the same phoneme. Currently I simply add all the keys which can result in strange behavior. I haven't figured out a good way to handle this yet.

I've been working on scrubbing + sound. My first try was awful since I don't have the control I need using MCI commands. I'm now trying making sound clip files on the fly and hope that's responsive enough.

Zooming + multiple tracks are also possibilities - but I first have a lot of learning to do on converting sampling rates and filtering or everything will be limited to a single sampling rate for each scene :'( (like sound output to AVI's is now limited).
Logged

johnar

  • Sr. Member
  • ****
  • Posts: 901
  • Make it, Move it--Give it Life
    • View Profile
    • youtube vids
Re: Lip Sync Tool
« Reply #29 on: Today at 01:16:39 am »

Quote
I'm also thinking about allowing the phonemes to be modified directly in the phoneme track. It takes a bit of experimenting to see what works smoothly.

 Seems to me that phonemes can already be edited directly in the P.track editor, and along with the graph editor, thats pretty well taken care of. (for me anyway). So that's interesting


Quote
Then there are issues with overlapping lead/tails of two adjacent instances of the same phoneme. Currently I simply add all the keys which can result in strange behavior. I haven't figured out a good way to handle this yet.

I'm trying to think of an occasion where i've ever needed to overlap the same phoneme keys, of the same name, in the timetrack.
  Very rarely.
 Perhaps a warning, like you get when adding keys to a sequence in Scene mode.?
 "you are about to overlap an existing phoneme of the same name, which will delete (the value) of the already existing phoneme key" 
'OK'  'CANCEL'
 Or maybe, rather than delete the existing key, the new one is added to the nearest frame possible after the existing one(s); (with a warning and OK/Cancel option)

Quote
I'm now trying making sound clip files on the fly and hope that's responsive enough.

 wow, that sounds really interesting.

Quote
Zooming + multiple tracks are also possibilities - but I first have a lot of learning to do on converting sampling rates and filtering or everything will be limited to a single sampling rate for each scene :'( (like sound output to AVI's is now limited)

May the force be with you steve.

 Personally, "being limited to a single sampling rate for each scene" isn't the end of the world. Its a small ask that the user limit .wav files to the same sampling rate per scene. (44.1kHz being standard)
 just my personal opinion tho.
 On the other hand, i can see how it could be important to handle different sampling rates

 Just know that, so far, everything 'Anim8or re lipsync' is coming along just dandy. Having lipsync abilities in Anim8or is already a thing of beauty.
Nice one steve
Logged
%
Pages: 1 [2] 3