How do I use wav2vec2-large-xlsr-53 instead of wav2vec2-chinese-base_fp16.safetensors?

by anujkk - opened Oct 19

Oct 19

How do I use wav2vec2-large-xlsr-53 instead of wav2vec2-chinese-base_fp16.safetensors? I tried converting it to safetensor and using it in the workflow but it gave error. Is it possible to use any other wav2vec2 model in WavVideoWrapper?

anujkk changed discussion title from How do I use wav2vec2-large-xlsr-53 instead of wav2vec2-chinese-base_fp16.safetensors to How do I use wav2vec2-large-xlsr-53 instead of wav2vec2-chinese-base_fp16.safetensors? Oct 19

Kijai

Owner Oct 20

Each audio model so far seems to be trained with specific version of wav2vec and doesn't work well or at all with different ones.

MattHVisual

Oct 20

I think the Chinese version works with any language, I've tested it with English, Spanish and people speaking English with various accents.

Kijai

Owner Oct 20

I think the Chinese version works with any language, I've tested it with English, Spanish and people speaking English with various accents.

This is true, it does, pretty sure it's finetuned on top of the English model.

mybumhurz

Nov 4

Hi, where in ComfyUI does the wav2vec belong?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment