Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Voice agent

A voice agent is an AI agent that holds a spoken conversation in real time, listening to a person, deciding what to say or do, and answering in a synthetic voice, often over a phone line or in an app.

Also called: voice AI agent

By Divya Sharma · Checked Nov 15, 2026 · How we review

Voice agents answer support lines, book appointments, qualify sales leads and act as hands-free assistants. They are judged less on what they know than on how natural the conversation feels: quick replies, no talking over the caller, and graceful recovery when interrupted.

How it works

The common design is a pipeline of three models. Speech-to-text turns the caller's audio into words, a language model decides the reply and calls any tools, and text-to-speech speaks the answer. Each stage streams, so the voice can start before the full reply is written. The newer design uses a single speech-to-speech model that takes audio in and gives audio out, which can cut delay and keep tone of voice, at the cost of less control over each step.

Around the models sit the hard parts: turn detection to know when the caller has finished, handling barge-in when they interrupt, telephony, and tool calls to calendars or CRMs. Frameworks such as LiveKit Agents and platforms such as Vapi and Retell AI package these pieces.

Why latency matters

People notice pauses in speech far more than in text. Every stage adds delay, so builders track time to first token for the model and time to first audio for the voice.

Limits

Background noise, accents and crosstalk cause transcription errors that the rest of the pipeline inherits. Many places have rules on disclosing that a caller is speaking to an AI and on recording calls; check the law where you operate.

Related terms

Speech-to-speech model
A speech-to-speech model takes spoken audio in and produces spoken audio out directly, without first converting the speech to text and back. It is used for live voice conversation and translation.
Text-to-speech
Text-to-speech (TTS) is software that turns written text into spoken audio. Modern TTS uses neural networks that predict the sound of speech from the text, then render it as a waveform.
Speech-to-text
Speech-to-text, also called automatic speech recognition (ASR), is software that converts spoken audio into written text. It powers transcription, captions, voice commands and the listening half of voice agents.
AI agent
An AI agent is a system in which a language model decides its own next steps: it calls tools, reads the results and repeats until the task is done, instead of following a fixed script.
Time to first token
Time to first token (TTFT) is how long a user waits between sending a request to a language model and receiving the first token of its reply, covering queuing, prompt processing and network time.
Voice cloning
Voice cloning is the use of AI to make a text-to-speech system speak in a specific person's voice, learned from a recording of that person. Some systems need only a short sample.

Sources