OSWorld
OSWorld is a benchmark for computer-use agents that runs real operating systems such as Ubuntu, Windows and macOS and scores agents on 369 open-ended tasks across web and desktop apps, checked by scripts.
Also called: OSWorld-Verified
By Divya Sharma · Checked Nov 15, 2026 · How we review
A computer-use agent controls a computer the way a person does: it looks at the screen, moves the mouse, clicks and types. OSWorld, published in 2024, tests these agents in real virtual machines rather than in a simplified mock-up.
How it works
The benchmark has 369 tasks drawn from real use. They involve web and desktop apps, file handling, and workflows that span several applications. Each task has a set-up configuration that puts the machine in a known starting state and a custom script that checks the final result. Because grading looks at what actually changed on the machine, an agent cannot pass by describing the right steps; it has to do them.
What the results showed
At launch, people completed over 72.36% of tasks while the best model managed 12.24%. The authors found agents struggled most with GUI grounding, working out which element on screen to act on, and with knowing how applications work. Launch figures date quickly, so check the current leaderboard.
Using the scores
OSWorld results depend on the model and on how the agent is built: the screenshots it sees, the tools it has and how many steps it may take. Compare like with like. For agents that only work in a web page, see browser agents. Many computer-use agents are built on vision-language models, so OSWorld also tests visual understanding. Run any such AI agent in an isolated environment, since it acts on a real system.
Related terms
- Computer use
- Computer use is the ability of an AI model to operate a computer the way a person does, by looking at screenshots and sending mouse clicks and keystrokes, so it can work with software that has no API.
- Browser agent
- A browser agent is an AI agent that controls a web browser to complete tasks, such as searching, clicking, filling in forms and reading pages, often running a headless browser with no visible window.
- AI benchmark
- An AI benchmark is a fixed, public set of tasks with an agreed scoring method, used to compare models on the same test, for example coding issues solved, exam questions answered or retrieval quality across many datasets.
- Vision-language model
- A vision-language model (VLM) is a model that takes images and text as input and produces text, so it can describe pictures, answer questions about them, read documents and extract data from screenshots.
- AI agent
- An AI agent is a system in which a language model decides its own next steps: it calls tools, reads the results and repeats until the task is done, instead of following a fixed script.