What is Foundry Local?
Foundry Local is Microsoft's runtime for running language and speech models on the user's own device. It is a roughly 20 MB package with native SDKs for C, C++, C#, JavaScript, Python and Rust, built on ONNX Runtime, that downloads models from a curated catalogue on first use, caches them and picks the best execution provider for the hardware: NPU, GPU or CPU.
The catalogue covers chat models such as GPT OSS, Qwen, DeepSeek, Mistral and Phi, plus Whisper for transcription, each quantised for consumer hardware and versioned so an app can pin a release. It speaks OpenAI request formats, including the Responses API, and has an optional OpenAI-compatible local web server and a preview CLI. It runs on Windows, macOS on Apple silicon and Linux, with no Azure subscription or API key.
The repository is active: v2.3.1 shipped on 3 November 2026, after v2.3.0 and the 0.13.0 CLI preview on 2 November. The SDKs are MIT-licensed; the CLI is under separate Microsoft licence terms.
Is Foundry Local worth it?
Foundry Local solves a different problem from Ollama or llama.cpp. Those run models for the person at the keyboard; Foundry Local is built to be bundled into an application you distribute, so it runs in your process through an SDK and adds about 20 MB to the package. Models download on first use, the runtime picks NPU, GPU or CPU, and the app works offline with no per-token bill.
That focus shapes the trade-offs. The catalogue is deliberately curated: Microsoft quantises and tests each model across consumer hardware, so you get fewer choices than Ollama's library but more predictable behaviour on unknown machines. Since v2.1.0 in September you can also register your own ONNX Runtime GenAI models. Six language SDKs and an OpenAI-compatible request format keep porting costs low.
The project ships fast. Version 2.2.1 on 23 October added tool calling for coding workflows, and 2.3.0 on 2 November improved coding-model support and fixed Parakeet and Whisper transcription. The repository has about 2,600 stars, small beside Ollama's 184,000, which means fewer community answers when something breaks.
Three defaults deserve a look before shipping. Telemetry is on by default; Microsoft says prompts, outputs, audio and raw device identifiers are not collected, and non-essential telemetry can be switched off in config or with ORT_DISABLE_TELEMETRY=1. CUDA and cuDNN runtimes are no longer bundled and must be installed separately. The optional local web server exposes an OpenAI-style endpoint that other processes on the machine can call, so leave it off in production builds unless you need it.
It is not a server stack. The README says it handles one user at a time, with no request queuing or continuous batching, and points multi-user workloads to vLLM or Triton. For a desktop or edge app that must keep data on the device, it is the cleanest option Microsoft offers.
Where it earns its place
A healthcare software vendor with a Windows desktop app
Summarise clinical notes without sending text to the cloud
Embed the C# SDK and a Phi or Qwen model from the catalogue; text stays on the clinician's PC, there is no per-token cost, and the app pins a model version so outputs do not change with an update.
A field-service company whose technicians work offline
Transcribe voice notes on laptops with no signal
Use the SDK's native Whisper transcription in Python or JavaScript; models are cached after the first download, so transcription works without connectivity.
A developer with an app already written against the OpenAI SDK
Offer a local mode without rewriting the client
Point the existing OpenAI client at Foundry Local's optional local server, which accepts OpenAI request formats; for distribution, move to the in-process SDK to avoid running a server.
Before you commit
- Setup
- Install the SDK (for example pip install foundry-local-sdk or npm install foundry-local-sdk), request a model from the catalogue, and it downloads, caches and loads in a few lines of code; winget installs the CLI on Windows.
- Integrations
- SDKs for C, C++, C#, JavaScript, Python and Rust; OpenAI-compatible formats including the Responses API; an optional local web server for LangChain and other tools.
- Data and privacy
- Inference runs on the device; telemetry is on by default, with prompts, outputs, audio and raw device identifiers excluded according to the privacy notes, and non-essential telemetry can be disabled.
- Cost at scale
- No licence or token fees; the cost is the user's hardware and the download size of each model.
- Limits
- Built for one user at a time, without request queuing or continuous batching; only catalogue or registered ONNX models; CUDA and cuDNN must be installed separately.
What’s good
- About 20 MB runtime that ships inside your app, with SDKs in six languages
- Automatic NPU, GPU or CPU selection through ONNX Runtime
- Curated, versioned model catalogue tested on consumer hardware
- OpenAI-compatible requests, including the Responses API format
- Frequent releases; v2.3.1 on 3 November 2026
What’s not
- Telemetry on by default
- Smaller catalogue than Ollama or llama.cpp
- Single-user design with no batching
- CUDA and cuDNN must be installed separately
- CLI is still a preview under non-MIT terms
Best for
Developers shipping desktop or edge apps that need a local model on users' own machines without an API key or cloud bill.
Foundry Local alternatives
Foundry Local: common questions
- Is Foundry Local worth it?
- Foundry Local is a tidy way to ship a small model inside a desktop app, but its curated catalogue, telemetry on by default and single-user design make it a poor substitute for Ollama on a workstation. We rate it 3.9 out of 5.
- How much does Foundry Local cost?
- Free. SDKs under MIT; the CLI under Microsoft's own licence terms; each model keeps its own licence. No Azure subscription or per-token charge.
- What are the downsides of Foundry Local?
- Telemetry on by default; smaller catalogue than Ollama or llama.cpp; single-user design with no batching; cUDA and cuDNN must be installed separately; cLI is still a preview under non-MIT terms.
- Who is Foundry Local best for?
- Developers shipping desktop or edge apps that need a local model on users' own machines without an API key or cloud bill.
- How much does Foundry Local cost for a team or in production?
- No licence or token fees; the cost is the user's hardware and the download size of each model.
- Is Foundry Local safe for company data?
- Inference runs on the device; telemetry is on by default, with prompts, outputs, audio and raw device identifiers excluded according to the privacy notes, and non-essential telemetry can be disabled.
- What are the best Foundry Local alternatives?
- The closest alternatives we have reviewed are Ollama, llama.cpp, llamafile and MLX.
Answers checked Nov 6, 2026.