Lip sync (AI video)
AI lip sync is the use of a model to change the mouth movements of a person in a video or image so they match a new audio track, such as a translated or rewritten line.
Also called: lip-sync, lipsync, AI lip sync, audio-driven talking face
By Divya Sharma · Checked Nov 15, 2026 · How we review
Lip sync is what makes AI dubbing look natural. Translate a speaker's words with text-to-speech or voice cloning, and the original mouth movements no longer fit. A lip sync model regenerates the mouth region so it matches the new speech.
How it works
Wav2Lip, from 2020, is a widely used open model. It edits the lower face in each frame to match the audio, and it was trained against a separate expert network that judges whether lips and sound are in sync. That judge pushed it to work on faces it had never seen, in ordinary unconstrained video. Newer models go further and animate the whole face or head from audio, which is the basis of an AI avatar.
Uses
Dubbing videos into other languages, fixing a line in post-production without a reshoot, personalised video messages and talking avatars. Services such as HeyGen and Synthesia build on it.
Limits
Teeth, side profiles, beards, fast head turns and hands near the face are hard. Results can look blurry around the mouth. Lip sync applied to a real person without consent is a core deepfake technique, so platform rules and laws may apply.
Related terms
- AI avatar
- An AI avatar is a generated video character, often based on a real person, that speaks a script or live responses with matching face, lip and head movements.
- Text-to-video
- Text-to-video is the task of generating a short video clip from a written prompt. Models must produce frames that each look right and also move consistently from one frame to the next.
- Deepfake
- A deepfake is image, audio or video content made or altered with AI so that it convincingly shows a real person, place or event in a way that did not happen and could pass as authentic.
- Voice cloning
- Voice cloning is the use of AI to make a text-to-speech system speak in a specific person's voice, learned from a recording of that person. Some systems need only a short sample.
- Text-to-speech
- Text-to-speech (TTS) is software that turns written text into spoken audio. Modern TTS uses neural networks that predict the sound of speech from the text, then render it as a waveform.