What it is
Phi-4-multimodal is a 5.6-billion-parameter model from Microsoft, announced on February 26, 2025. It adds speech and vision encoders to the Phi-4-mini language model using a mixture of LoRAs (small add-on weight sets), so one model takes text, images and audio and writes text. It has a 128K-token context.
Text works in 23 languages, speech in 8 (English, Chinese, German, French, Italian, Japanese, Spanish and Portuguese), and vision is English only. Its training data runs to June 2024. Microsoft suggests keeping audio to 40 seconds for transcription and 30 minutes for summarisation.
The weights are under MIT on Hugging Face, with an official ONNX build. At launch, Azure AI Foundry charged $0.00008 per 1,000 input tokens and $0.00032 per 1,000 output tokens for text and images, and $0.004 per 1,000 audio input tokens. In January 2026 Microsoft published Paza, a research-only fine-tune for speech recognition in six East African languages.
Our review
Phi-4-multimodal is still Microsoft's only open model that hears speech, and that is now its main argument. At launch Microsoft said it topped the Hugging Face OpenASR leaderboard with a 6.14% word error rate, against a previous best of 6.5%, and its average on Microsoft's vision tests was 72.0, close to GPT-4o's 72.4 in the same table. Both figures are self-reported. Microsoft also conceded a gap to Gemini 2.0 Flash and GPT-4o's realtime model at answering spoken questions, because a small model retains less factual knowledge.
Microsoft's later Phi work went elsewhere. Phi-4-reasoning-vision-15B, from January 2026, reads images but not audio. The only speech follow-up is Paza, a fine-tune that trained just the audio components for Swahili, Kalenjin, Kikuyu, Luo, Maasai and Somali, which Microsoft labels a research release not recommended for commercial use without further testing.
The competition has moved. Google's Gemma 4 E2B and E4B, released in April 2026 under Apache 2.0, also take audio and images at a phone-friendly size. Phi-4-multimodal's vision is English only, and Microsoft's red-teaming found it more susceptible to context manipulation and persuasion than it would like, with persuasion working best in French and Italian.
The operational details deserve more attention than the leaderboard number. Transcription quality is only promised for clips up to 40 seconds, flash attention needs an Ampere-class GPU or newer, and vLLM needs the speech and vision LoRA folders passed in separately. Anything longer than a voicemail has to be segmented before it reaches the model, and that plumbing needs testing before any word error rate means much.
Where it earns its place
A mobile developer building an offline field-inspection app
Turn a short spoken note and a photo into one structured report on the device
One set of weights reads both inputs, and an ONNX build exists for on-device runtimes. Clips should stay under 40 seconds, and Microsoft warns accuracy falls for accents other than standard American English.
A contact-centre analytics lead at a European insurer
Summarise recorded calls in German, French or Spanish without sending audio to a cloud API
Self-hosted on an A100 or H100, which Microsoft recommends, it accepts up to 30 minutes of audio for summarisation. The MIT licence has no revenue cap, but the persuasion findings argue for checking outputs before they reach customers.
A speech researcher working on an under-served language
Adapt speech recognition to a new language without training a model from scratch
Microsoft's own Paza fine-tune unfroze only the audio encoder and projection layers, leaving the language model intact. The card's speech fine-tuning example ran on a single A100 80GB.
Before you commit
- Setup
- Load from Hugging Face with Transformers and trust_remote_code. Flash attention needs an Ampere-class or newer GPU; V100s must fall back to eager attention.
- Integrations
- Transformers and vLLM's OpenAI-compatible server (with separate speech and vision LoRA folders), an official ONNX build, Azure AI Foundry and the NVIDIA API catalogue.
- Data and privacy
- MIT licence with no revenue cap or separate use policy. Self-hosted, audio and images stay on your hardware; Azure's terms apply only if you use Microsoft's hosted endpoint.
- Cost at scale
- At launch prices on Azure AI Foundry, audio input cost $4 per 1M tokens against $0.08 for text and images, so speech dominates any bill; output was $0.32 per 1M. Current Foundry pricing was not re-verified. Self-hosting needs one recent data-centre GPU.
- Limits
- 128K-token context, but audio is advised at 40 seconds for transcription and 30 minutes for summarisation. Weaker at spoken question answering than larger models, by Microsoft's own account.
What's good
- Speech, image and text input in one 5.6B model
- MIT licence with no revenue cap
- 6.14% OpenASR word error rate at launch (self-reported)
- 128K-token context; official ONNX build
What's not
- Released February 2025, training data to June 2024
- Vision is English only, speech covers 8 languages
- Transcription advised for clips of 40 seconds or less
- More setup work to serve than a plain text model
Best for
Developers who want a small MIT-licensed model for on-device speech and image input.
Alternatives
Phi-4-multimodal: common questions
- Is Phi-4-multimodal worth it?
- Still the only Phi model that hears speech, but at 20 months old it has been overtaken by newer small multimodal models. We rate it 3.3 out of 5.
- How much does Phi-4-multimodal cost?
- Free weights under MIT. Azure AI Foundry at launch: $0.08 input / $0.32 output per 1M text or image tokens; audio input $4 per 1M.
- What are the downsides of Phi-4-multimodal?
- Released February 2025, training data to June 2024; vision is English only, speech covers 8 languages; transcription advised for clips of 40 seconds or less; more setup work to serve than a plain text model.
- Who is Phi-4-multimodal best for?
- Developers who want a small MIT-licensed model for on-device speech and image input.
- How much does Phi-4-multimodal cost for a team or in production?
- At launch prices on Azure AI Foundry, audio input cost $4 per 1M tokens against $0.08 for text and images, so speech dominates any bill; output was $0.32 per 1M. Current Foundry pricing was not re-verified. Self-hosting needs one recent data-centre GPU.
- Is Phi-4-multimodal safe for company data?
- MIT licence with no revenue cap or separate use policy. Self-hosted, audio and images stay on your hardware; Azure's terms apply only if you use Microsoft's hosted endpoint.
- What are the best Phi-4-multimodal alternatives?
- The closest alternatives we have reviewed are Gemma 4, Phi-4-Reasoning-Vision-15B and Whisper large-v3-turbo.
Answers checked Oct 22, 2026.