What is audio.cpp?
audio.cpp is a native C++ inference engine for audio models, built on ggml (the tensor library behind llama.cpp) by ShugoAI LLC and released under Apache 2.0. One CLI, one HTTP server and an embedded web UI cover text to speech, voice cloning and conversion, speech recognition, voice activity detection, diarisation, source separation, alignment and music generation, with no Python needed at inference time. Version 0.9.2, released on 19 October 2026, lists more than 110 model families and 190 variants, from Qwen3-TTS and VibeVoice to Whisper-class ASR, Demucs and ACE-Step.
The repository was created in June 2026 and has shipped 23 releases since. It runs on CUDA, HIP/ROCm, Vulkan, Metal and CPU, with prebuilt binaries for Windows, Ubuntu and macOS, Docker images and a Homebrew tap. Unsloth Studio now uses it for speech, music and transcription. Each model keeps its original licence, and many of the supported weights are non-commercial.
Is audio.cpp worth it?
audio.cpp replaces a drawer full of Python environments with one native binary, and that alone justifies it for anyone running more than two or three audio models. Each speech model normally arrives with its own Conda stack and pinned PyTorch version; here 110-plus families load through the same CLI, the same GGUF packaging and the same server, which exposes /v1/audio/speech, /v1/audio/transcriptions, alignment and batch task endpoints. A JSON pipeline layer chains them, so a redub job can transcribe with Qwen3-ASR and resynthesise with Qwen3-TTS in one command.
The speed claims are the maintainer's, measured on an RTX 5090 against each model's Python reference. Selected CUDA TTS paths run 1.8 to 8 times faster, and Q8 GGUF packages are reported at up to 1.53 times faster with 37% lower peak VRAM than 16-bit weights. The project is candid that quantisation is not a free lunch: on Qwen3-TTS q8_0 bought only about 3.8% more speed, and on Vevo2 non-fp32 output drifted measurably from the reference. An independent meeting-transcription benchmark, TranscrIA, found its Nemotron ASR port matched other implementations' word error rate in about a quarter of the wall time.
The licence question sits one level down. The engine is Apache 2.0, but docs/model_licenses.md marks many supported weights as non-commercial, including Breeze TTS 2, Higgs Audio TTS, F5-TTS and CrisperWhisper, and some as unclear. A team that picks models from the web UI's catalogue can end up shipping a product on weights it may not use commercially, so the licence table is required reading.
CUDA is the optimised path; Vulkan, Metal, HIP and CPU are described as portability backends with lower coverage, and AMD Windows builds come from a community fork. The server keeps every model it has loaded in memory unless max_loaded_models is set, and some model preparation still calls a Python model manager. Its GGUF files are not interchangeable with llama.cpp or whisper.cpp ones. For a project this young, with 15 open issues and pull requests against 3,775 stars, the cadence is high and the surface still moving.
Where it earns its place
A platform engineer at a 50-person call-analytics company
Transcribe and diarise recorded calls on one in-house GPU box
Run audiocpp_server with Nemotron or Qwen3 ASR and a diarisation model, call /v1/audio/transcriptions/details, and cap residency with max_loaded_models. Hardware is the only cost; check each model's licence row before production.
A desktop-app developer shipping offline voiceover on Windows and macOS
Bundle local TTS without asking users to install Python
Link the C API or ship the prebuilt binaries with a small model such as Pocket TTS, which the maintainer measured at about 31 times real time on CUDA. The yovoice and OpenDub apps already take this route.
A video localisation studio dubbing training courses
Redub long recordings into a target voice
Run the bundled speech_redub pipeline, which splits long audio, transcribes with Qwen3-ASR and regenerates in the reference voice with Qwen3-TTS; the test input is a seven-minute speech. Apache-licensed Qwen models keep the output commercially usable.
Before you commit
- Setup
- Download a release binary, docker run the CUDA or CPU image, or brew install on macOS; then audiocpp_server --ui --ui-management opens a web UI on port 8080 that downloads models. Building from source needs GCC 13 or MSVC 2022 plus the backend SDK.
- Integrations
- HTTP endpoints for speech, transcription, alignment and batch tasks; a C ABI with community .NET and Python bindings; JSON pipelines; Unsloth Studio, Project AIRI and pyVideoTrans integrate it.
- Data and privacy
- Runs fully locally with no telemetry described in the README. --ui-management enables downloads, uploads and model switching, so keep that mode off on shared networks. Engine is Apache 2.0; each model's weights keep their own licence.
- Cost at scale
- No paid tier; the cost is GPU time. Memory depends on the model, and a loaded model stays resident until the server exits unless max_loaded_models is set.
- Limits
- CUDA is the optimised backend; others have lower performance and coverage. Quantisation can break some paths (ACE-Step fails at low precision) or degrade quality, and GGUF files from llama.cpp or whisper.cpp are not compatible.
What's good
- 110+ audio model families behind one CLI, server and web UI
- No Python at inference; CUDA, Vulkan, Metal, HIP and CPU backends
- Prebuilt binaries, Docker images, Homebrew and a C API
- Published per-model licence table flags non-commercial weights
What's not
- Speed figures are the maintainer's, measured on an RTX 5090
- CUDA is the only fully optimised backend
- Many supported weights are non-commercial
- Created June 2026; formats and options still changing release to release
Best for
Engineers who self-host several speech or audio models and want one native runtime and HTTP API instead of a Python environment per model.
audio.cpp alternatives
audio.cpp: common questions
- Is audio.cpp worth it?
- audio.cpp is doing for local speech what llama.cpp did for chat models, and four months in it already runs more than 110 model families. We rate it 4.3 out of 5.
- How much does audio.cpp cost?
- Free under Apache 2.0. No paid edition. Model weights keep their own licences, many of them non-commercial.
- What are the downsides of audio.cpp?
- Speed figures are the maintainer's, measured on an RTX 5090; cUDA is the only fully optimised backend; many supported weights are non-commercial; created June 2026; formats and options still changing release to release.
- Who is audio.cpp best for?
- Engineers who self-host several speech or audio models and want one native runtime and HTTP API instead of a Python environment per model.
- How much does audio.cpp cost for a team or in production?
- No paid tier; the cost is GPU time. Memory depends on the model, and a loaded model stays resident until the server exits unless max_loaded_models is set.
- Is audio.cpp safe for company data?
- Runs fully locally with no telemetry described in the README. --ui-management enables downloads, uploads and model switching, so keep that mode off on shared networks. Engine is Apache 2.0; each model's weights keep their own licence.
- What are the best audio.cpp alternatives?
- The closest alternatives we have reviewed are llama.cpp, Ollama and llamafile.
Answers checked Oct 26, 2026.
Make audio.cpp? Show this rating on your site
