Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

OSWorld

OSWorld is a benchmark for computer-use agents that runs real operating systems such as Ubuntu, Windows and macOS and scores agents on 369 open-ended tasks across web and desktop apps, checked by scripts.

Also called: OSWorld-Verified

By Divya Sharma · Checked Nov 15, 2026 · How we review

A computer-use agent controls a computer the way a person does: it looks at the screen, moves the mouse, clicks and types. OSWorld, published in 2024, tests these agents in real virtual machines rather than in a simplified mock-up.

How it works

The benchmark has 369 tasks drawn from real use. They involve web and desktop apps, file handling, and workflows that span several applications. Each task has a set-up configuration that puts the machine in a known starting state and a custom script that checks the final result. Because grading looks at what actually changed on the machine, an agent cannot pass by describing the right steps; it has to do them.

What the results showed

At launch, people completed over 72.36% of tasks while the best model managed 12.24%. The authors found agents struggled most with GUI grounding, working out which element on screen to act on, and with knowing how applications work. Launch figures date quickly, so check the current leaderboard.

Using the scores

OSWorld results depend on the model and on how the agent is built: the screenshots it sees, the tools it has and how many steps it may take. Compare like with like. For agents that only work in a web page, see browser agents. Many computer-use agents are built on vision-language models, so OSWorld also tests visual understanding. Run any such AI agent in an isolated environment, since it acts on a real system.

Related terms

Computer use
Computer use is the ability of an AI model to operate a computer the way a person does, by looking at screenshots and sending mouse clicks and keystrokes, so it can work with software that has no API.
Browser agent
A browser agent is an AI agent that controls a web browser to complete tasks, such as searching, clicking, filling in forms and reading pages, often running a headless browser with no visible window.
AI benchmark
An AI benchmark is a fixed, public set of tasks with an agreed scoring method, used to compare models on the same test, for example coding issues solved, exam questions answered or retrieval quality across many datasets.
Vision-language model
A vision-language model (VLM) is a model that takes images and text as input and produces text, so it can describe pictures, answer questions about them, read documents and extract data from screenshots.
AI agent
An AI agent is a system in which a language model decides its own next steps: it calls tools, reads the results and repeats until the task is done, instead of following a fixed script.

Sources