What is VibeVoice?
VibeVoice is Microsoft Research's open repository for a family of speech models that share one design: continuous acoustic and semantic tokenisers running at 7.5 frames a second, a Qwen2.5 language model for context and a diffusion head for audio detail. The repo holds inference code, demos, vLLM serving guides and ASR fine-tuning scripts; the weights live on Hugging Face. Everything is MIT-licensed.
Five models are listed. VibeVoice-ASR (January 2026) transcribes up to 60 minutes in one pass with speaker labels and timestamps in 50+ languages. VibeVoice-ASR-Streaming (September 2026) emits text while audio arrives, in 10 languages. VibeVoice-ASR-BitNet (July 2026) is a 1.58 GB CPU build served by a separate repo, VibeASR.cpp. VibeVoice-Realtime-0.5B (December 2025) is a single-speaker streaming text-to-speech model. VibeVoice-TTS-1.5B (August 2025) generates up to 90 minutes of conversation with up to four speakers, but its code was removed on 5 September 2025 after misuse.
The repo was created on 25 August 2025, has about 56,600 stars and 6,375 forks, and publishes no GitHub releases; the last commit landed on 8 October 2026. The README calls every model 'intended for research and development purposes only'.
Is VibeVoice worth it?
VibeVoice is worth following for its recognition models, not its speech synthesis. The flagship is VibeVoice-ASR, reviewed separately as a model, which takes an hour of audio in one 64K-token window and returns who spoke, when and what they said. Around it the repo has grown a streaming variant, a vLLM serving path, fine-tuning code and a 1.58 GB BitNet build that Microsoft says runs faster than real time on three CPU threads, compressed from 4.62 GB. Hugging Face lists the ASR weights at about 8.7 billion parameters, so the full model wants a data-centre GPU.
The text-to-speech history shapes how far to trust the rest. VibeVoice-TTS-1.5B, about 2.7 billion parameters on Hugging Face despite the name, was released on 25 August 2025 with 90-minute, four-speaker generation and was accepted as an oral at ICLR 2026. Eleven days later Microsoft removed the code, saying the tool had been 'used in ways inconsistent with the stated intent'. Hacker News users watched the repo go private and return a 404 before it came back; the larger 7B TTS weights were withdrawn from Hugging Face and the hosted demo is disabled. The 1.5B weights are still downloadable, with an audible AI disclaimer and an imperceptible watermark built into every output, and a community fork keeps the removed code alive. That is outside Microsoft's support and its research-only terms.
VibeVoice-Realtime-0.5B is the TTS Microsoft still ships code for, and it is deliberately narrow. It speaks with one voice at a time, English first, with nine experimental languages; voices come as embedded prompts, so there is no cloning from a sample. First audio arrives in about 200 to 300 milliseconds depending on which page of Microsoft's docs you read, and an 8K context caps a single generation at about 10 minutes. Microsoft reports a 2.00% word error rate on LibriSpeech test-clean and says an NVIDIA T4 or Apple M4 Pro keeps up in real time.
The licence and the caveat pull in opposite directions. MIT permits commercial use, yet the README and model cards say the models are for research and development and advise against commercial deployment without further testing. The install guides start from NVIDIA PyTorch containers launched with --privileged and --net=host, which is fine on a dev box and wrong on a shared server, and the streaming ASR demo serves a WebSocket page on port 7870. The latest commit, on 8 October 2026, removed remote audio URL fetching from streaming ASR. With 205 open issues and pull requests and no tagged releases, pinning a commit is the only way to know what you are running.
Where it earns its place
An ML engineer at a 300-person company recording internal meetings
Run batch and live transcription from one model family on in-house GPUs
Serve VibeVoice-ASR with vLLM for recordings of up to an hour and VibeVoice-ASR-Streaming for live captions, passing attendee names as hotwords to both. Streaming covers 10 languages against 50+ for batch, so check the language list first.
A developer building a voice front end for an LLM agent
Make the agent start speaking before its answer is finished
Feed streamed text into VibeVoice-Realtime-0.5B, which Microsoft says produces first audio in about 200 to 300 milliseconds and runs in real time on a T4. Only built-in voices are available, and anything beyond English is experimental.
A speech researcher studying long-form dialogue synthesis
Reproduce the multi-speaker TTS results from the ICLR 2026 paper
The 1.5B weights and the paper remain public, but Microsoft no longer ships inference code for them; the README links the technical report, and every output carries an audible disclaimer and a watermark.
Before you commit
- Setup
- Start the NVIDIA PyTorch container the docs name (24.07 to 25.12 verified), install flash attention if missing, then pip install -e . from a clone; the realtime TTS needs the [streamingtts] extra and the streaming ASR needs ffmpeg. VibeVoice-ASR also loads natively in Transformers.
- Integrations
- Hugging Face Transformers for ASR, vLLM serving guides for batch and streaming ASR, VibeASR.cpp for CPU, Azure AI Foundry Labs for a hosted trial, Gradio and FastAPI demos, and a Colab notebook for the realtime TTS.
- Data and privacy
- Self-hosted, so audio and text stay on your hardware. The suggested docker commands use --privileged and --net=host, and the streaming ASR demo listens on port 7870; keep both off shared networks. MIT licence, but the README describes research and development use only.
- Cost at scale
- No hosted price. The cost is GPU time: about 8.7 billion parameters for ASR, about 2.7 billion for the 1.5B TTS and about 1 billion for the 0.5B realtime model as Hugging Face counts them, or CPU time for the 1.58 GB BitNet build.
- Limits
- ASR takes 60 minutes per pass; realtime TTS about 10 minutes per generation, one speaker, English first. Long-form TTS has no official inference code. Benchmarks are Microsoft's own, and there are no tagged releases to pin.
What’s good
- Long-form ASR with speaker labels and timestamps in one pass
- Streaming ASR, vLLM serving and fine-tuning code in the same repo
- 1.58 GB BitNet ASR build runs on CPU
- MIT licence on code and weights
What’s not
- Long-form TTS code removed in September 2025; 7B TTS weights withdrawn
- Realtime TTS is single-speaker, English-first and has no voice cloning
- README marks every model research and development only
- No tagged releases; 205 open issues and pull requests
Best for
Teams that self-host transcription of long multi-speaker audio and want one maintained repo for batch, streaming and CPU recognition, plus a small real-time English voice.
VibeVoice alternatives
VibeVoice: common questions
- Is VibeVoice worth it?
- VibeVoice is now a strong open transcription line with a small real-time voice attached, and the long-form TTS that made it famous is the one part you can no longer run from Microsoft's code. We rate it 3.9 out of 5.
- How much does VibeVoice cost?
- Free under the MIT licence; weights free on Hugging Face. VibeVoice-ASR is also in Azure AI Foundry Labs. No paid edition.
- What are the downsides of VibeVoice?
- Long-form TTS code removed in September 2025; 7B TTS weights withdrawn; realtime TTS is single-speaker, English-first and has no voice cloning; rEADME marks every model research and development only; no tagged releases; 205 open issues and pull requests.
- Who is VibeVoice best for?
- Teams that self-host transcription of long multi-speaker audio and want one maintained repo for batch, streaming and CPU recognition, plus a small real-time English voice.
- How much does VibeVoice cost for a team or in production?
- No hosted price. The cost is GPU time: about 8.7 billion parameters for ASR, about 2.7 billion for the 1.5B TTS and about 1 billion for the 0.5B realtime model as Hugging Face counts them, or CPU time for the 1.58 GB BitNet build.
- Is VibeVoice safe for company data?
- Self-hosted, so audio and text stay on your hardware. The suggested docker commands use --privileged and --net=host, and the streaming ASR demo listens on port 7870; keep both off shared networks. MIT licence, but the README describes research and development use only.
- What are the best VibeVoice alternatives?
- The closest alternatives we have reviewed are audio.cpp and VoiceStudio.
Answers checked Nov 7, 2026.