Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Voice cloning

Voice cloning is the use of AI to make a text-to-speech system speak in a specific person's voice, learned from a recording of that person. Some systems need only a short sample.

Also called: voice clone, voice replication, custom voice

By Divya Sharma · Checked Nov 15, 2026 · How we review

A cloned voice lets a text-to-speech model read any text so it sounds like the person in the sample. Uses include dubbing, audiobooks narrated by the author, restoring the voice of someone who has lost the ability to speak, and branded voice agents.

How it works

Baidu researchers described the two main approaches in 2018. Speaker adaptation fine-tunes a multi-speaker model on a few clips of the new voice. Speaker encoding trains a separate network that turns a clip into a speaker embedding, a compact description of the voice, which the speech model then follows. Encoding needs no extra training per voice, so it is faster and cheaper. Modern zero-shot TTS models extend the encoding idea. Hosted services such as ElevenLabs and open models such as Chatterbox offer cloning.

Consent and misuse

A convincing clone can be used for fraud, such as a fake phone call from a relative or an executive, or to put words in a public figure's mouth. Cloned audio combined with video is a common form of deepfake. Many providers ask users to confirm they have the speaker's permission, and some add audio watermarks. Laws on using someone's voice differ by country, so check the rules that apply to you.

Limits

Clones from short samples can miss accent, emotion and breathing patterns, and quality drops with noisy source audio.

Related terms

Text-to-speech
Text-to-speech (TTS) is software that turns written text into spoken audio. Modern TTS uses neural networks that predict the sound of speech from the text, then render it as a waveform.
Deepfake
A deepfake is image, audio or video content made or altered with AI so that it convincingly shows a real person, place or event in a way that did not happen and could pass as authentic.
AI watermarking
AI watermarking is embedding a hidden, machine-detectable signal in text, images, audio or video at the moment an AI model generates it, so software can later identify the content as AI-generated.
Speech-to-speech model
A speech-to-speech model takes spoken audio in and produces spoken audio out directly, without first converting the speech to text and back. It is used for live voice conversation and translation.
Voice agent
A voice agent is an AI agent that holds a spoken conversation in real time, listening to a person, deciding what to say or do, and answering in a synthetic voice, often over a phone line or in an app.

Sources