What is UFO?
UFO is an open-source agent framework from Microsoft Research that operates Windows applications for you. Given a request in plain language, it takes screenshots, reads the Windows UI Automation tree, and then clicks, types and calls app APIs in the real programs on your machine: Office, Edge, File Explorer and the rest. It is not a sandbox; it acts on whatever desktop it runs on.
The repository holds two layers. UFO² (April 2025) is the Windows agent itself, now in long-term support. UFO³ Galaxy (November 2025) splits a task into a graph of steps and hands them to agents on Windows, Linux and Android devices over a WebSocket protocol; Microsoft labels it active development, suited to experiments and non-critical work.
It needs Windows 10 or later, Python 3.10 or later and an API key for a model provider: OpenAI, Azure OpenAI, Claude, Gemini, Qwen, DeepSeek or a local model through Ollama. The code is MIT-licensed, has about 10,200 GitHub stars, and v3.0.10 was released on 22 September 2026.
Is UFO worth it?
UFO's advantage over screenshot-only agents is that it reads Windows itself. UFO² combines the UI Automation accessibility tree and Win32 and COM calls with vision, so it can press a button by name or call an Office API instead of guessing pixel positions. In Microsoft's paper that lifts results clearly: with o1 it completed 30.5% of Windows Agent Arena tasks and 32.7% of OSWorld-W, against 20.8% and 14.3% for OpenAI's Operator, with every agent capped at 30 steps.
Those are still failure rates of roughly 70%, and the agent acts on the desktop you are using. It moves the pointer, types into documents and email, and sends screenshots of your screen to the model provider you configured; Microsoft's disclaimer tells users to keep sensitive information off screen and to run it in a controlled environment. A SAFE_GUARD setting, on by default, asks for confirmation before actions the model marks as sensitive, but the same document admits the gate depends on the model doing the marking and is not a sandbox. A dedicated Windows virtual machine is the sensible home for it.
Security work has been steady and is worth tracking. Releases v3.0.8 and v3.0.9, in August and September 2026, fixed several issues reported to Microsoft's security response centre, including server-side request forgery through the Galaxy web UI. On 29 September a fix landed so that sensitive actions no longer ran before the confirmation gate; that came a week after v3.0.10, so the latest tagged release does not include it.
Cost is the other trade. Each step is a model call with a screenshot attached; the speculative multi-action feature batches predicted steps and, per the README, cuts LLM calls by up to 51%. Galaxy adds cross-device planning, but it is younger and harder to set up. For browser-only work, browser-use is lighter and easier to contain; UFO earns its place when the job lives inside Windows desktop software that has no API.
Where it earns its place
An RPA developer at an insurer running legacy Windows claims software
Prototype automating a data-entry routine that has no API
Run UFO² in a dedicated Windows VM with SAFE_GUARD on, add a recorded demonstration to its retrieval knowledge base, and measure how often it completes the routine end to end. At benchmark success rates near 30%, plan for a person checking every run before it touches production records.
A research group studying computer-use agents
Compare models on the same Windows tasks
Swap the model in agents.yaml between GPT-4o, Claude, Gemini or a local Ollama model and run the Windows Agent Arena integration Microsoft documents. The paper's 30-step cap and per-model tables give a published baseline to compare against.
An IT team at a 300-person firm with Windows laptops and Linux servers
Test a workflow that pulls a log from a server and writes a summary into an Office document
Galaxy splits the request into a graph, sending the Linux step to a Linux device agent and the Office step to UFO² on Windows. Microsoft calls Galaxy suitable for experimentation and non-critical workflows, so keep it to a pilot.
Before you commit
- Setup
- On Windows 10 or later with Python 3.10 or later, pip install -r requirements.txt, copy config/ufo/agents.yaml.template, add an API key, then run python -m ufo --task <name>. Galaxy also needs a devices.yaml and a server and client running on each device.
- Integrations
- OpenAI, Azure OpenAI, Claude, Gemini, Qwen, DeepSeek, Ollama and OpenAI's Operator model; MCP servers for device-agent tools; Bing search and retrieval over documents and demonstrations; Windows Agent Arena and OSWorld for evaluation.
- Data and privacy
- Screenshots of the desktop and active applications go to the configured model provider at every step; Microsoft says it does not collect them, but the provider's terms apply. API keys sit in config/ufo/agents.yaml. The agent controls the real mouse and keyboard, so run it in a dedicated VM or account; the SAFE_GUARD confirmation relies on the model flagging sensitive actions.
- Cost at scale
- Free under MIT; spending is model API usage, one or more vision calls per step, with multi-action batching cutting calls by up to 51% per the README. Microsoft does not publish a cost per task.
- Limits
- Best published success rates are 30.5% on Windows Agent Arena and 32.7% on OSWorld-W; UFO² runs on Windows only; Galaxy is in active development; the 29 September 2026 confirmation-gate fix is newer than v3.0.10.
What’s good
- Uses Windows UI Automation, Win32 and COM, not just screenshots
- 30.5% on Windows Agent Arena with o1, above Operator's 20.8%
- Works with OpenAI, Azure OpenAI, Claude, Gemini, Qwen, DeepSeek or Ollama
- MIT licence, with regular security fixes through 2026
What’s not
- Clicks and types in your real apps; it is not sandboxed
- Sends screenshots of your desktop to the model provider
- Fails roughly 70% of benchmark tasks even at its best
- UFO² is Windows-only; Galaxy is still in active development
Best for
Automation engineers in Windows-heavy organisations who want to prototype agents for desktop software with no API, inside a test virtual machine.
UFO alternatives
UFO: common questions
- Is UFO worth it?
- UFO shows how far deep Windows integration can push a desktop agent, but it still fails most benchmark tasks while driving your real mouse and keyboard. We rate it 3.5 out of 5.
- How much does UFO cost?
- Free and open source under the MIT licence; you pay your model provider for every step.
- What are the downsides of UFO?
- Clicks and types in your real apps; it is not sandboxed; sends screenshots of your desktop to the model provider; fails roughly 70% of benchmark tasks even at its best; uFO² is Windows-only; Galaxy is still in active development.
- Who is UFO best for?
- Automation engineers in Windows-heavy organisations who want to prototype agents for desktop software with no API, inside a test virtual machine.
- How much does UFO cost for a team or in production?
- Free under MIT; spending is model API usage, one or more vision calls per step, with multi-action batching cutting calls by up to 51% per the README. Microsoft does not publish a cost per task.
- Is UFO safe for company data?
- Screenshots of the desktop and active applications go to the configured model provider at every step; Microsoft says it does not collect them, but the provider's terms apply. API keys sit in config/ufo/agents.yaml. The agent controls the real mouse and keyboard, so run it in a dedicated VM or account; the SAFE_GUARD confirmation relies on the model flagging sensitive actions.
- What are the best UFO alternatives?
- The closest alternatives we have reviewed are Browser Use, CLI-Anything, Magentic-UI (MagenticLite) and Agent Zero.
Answers checked Nov 14, 2026.