ComfyUI Multilingual Lip-Sync Dubbing Workflow Tutorial
Start here
The ComfyUI team has published a tutorial demonstrating a multilingual lip-sync dubbing workflow that can replace the language within the same shot, preserve the actor's cloned voice, and repaint only the mouth area. The process first uses ElevenLabs to separate and clone the voice and generate word-level timestamps, then uses an LLM to translate while matching the original duration, and finally uses MiniMax H3 to repaint within the mouth mask and composite the result back into the original footage. The workflow is suitable for 24 fps, a single speaker, and about 15 seconds per run. The output contains only the dubbed audio, so ambient sound and music must be added back separately. The dubbing may end about 0.5 seconds early, but the lip movements remain synchronized. Each language takes about 10–12 minutes on Comfy Cloud.
Written by AI from the reports · updated 1 hour ago
Report timeline
Follow the reports to see the event from other sides.
- comfyorg · YouTube 创作视频PicksComfyUI Multilingual AI Lip-Sync Dubbing Workflow Tutorial
The ComfyUI team demonstrates a multilingual AI lip-sync dubbing workflow that can replace the language within the same shot, preserve the actor's cloned voice, and repaint only the mouth area. The process first uses ElevenLabs to separate and clone the voice and generate word-level timestamps, then uses an LLM to translate while matching the original duration, and finally uses MiniMax H3 to repaint within the mouth mask and composite the result back into the original footage. The workflow supports 24 fps, a single speaker, and about 15 seconds per run, taking about 10–12 minutes per language on Comfy Cloud.
Heat trend for this event
Not enough consecutive observations to draw a trend yet.