What is ARTEMIS?
ARTEMIS is an Android automation agent from Google's Pixel Test Engineering team. You describe a task in plain language, such as "open Settings, find Battery and report the level", and it works the phone or emulator over ADB: reading the screen through an accessibility helper it installs on the device, falling back to OCR and visual models for custom interfaces, tapping and typing, and recording logs, screenshots and a replay.
There are four ways in: a web console at localhost:8000 with live screen mirroring, a CLI, a Python SDK for pytest and CI, and an MCP server that the setup script can register in Claude Code, Codex, Cursor, Windsurf, VS Code, Antigravity and others, together with a rules file that tells the coding agent how to test. Two profiles trade speed for care: Flash, a single-model observe-and-act loop at about 3 to 5 seconds a step, and Pro, a planner, operator and checker graph at about 15 to 40 seconds a step.
It is Apache-2.0 and written in Python 3.12. You supply the model: Gemini, OpenAI, Anthropic, OpenRouter or xAI keys are all accepted.
Is ARTEMIS worth it?
ARTEMIS gives coding agents something they have lacked: hands on a real phone. Point Claude Code or Codex at it through MCP and the agent can build an APK, install it, walk through a login and come back with screenshots and Logcat output, which is most of what a human tester does when reproducing a bug.
The headline claim is a completion rate above 99% on AndroidWorld, Google Research's benchmark of 100+ multi-step tasks across 20+ apps. That figure is the project's own and has not been reproduced independently, and AndroidWorld's apps are friendlier than a Flutter or Compose screen with custom widgets. Adoption is not in doubt: the repository passed 16,401 stars and 1,802 forks within three months of its August 2026 launch, though it has only six contributors and no tagged releases.
The maintainers are candid about where it fits. Asked in the project's discussions about running thousands of regression tests, they argued against compiling agent runs into tests: a workflow that finds its way around a changed screen can hide the very bug a test exists to catch. Their recommended loop is to explore with ARTEMIS, have a coding agent write ordinary test code from what it learned, and run that in your existing framework, calling ARTEMIS from the SDK only for awkward steps such as Bluetooth pairing.
The costs are practical ones. Every step is a model call, so a 40-step Pro run at 15 to 40 seconds a step takes 10 to 27 minutes and bills accordingly. The first task installs an accessibility service on the phone, which is fine for a test device and worth thinking about on a personal one; the .env file has switches to stop that and to let the screen sleep. iOS is on the roadmap, not in the code. Part of the code comes from Minitap's mobile-use project, credited under third_party/.
Where it earns its place
An Android developer at a five-person app studio
Reproduce a crash report without clicking through the app myself
Run artemis mcp --install claude, then ask Claude Code to build the APK, install it and follow the report's steps; ARTEMIS returns screenshots, Logcat and a replay. Cost is the model calls, a few seconds each in Flash mode.
A QA lead at a fintech with a pytest suite
Script the flaky steps that change on every OS update
Call ArtemisClient.run() from the SDK for the Bluetooth pairing or permission dialogs only, assert on its typed result, and keep the rest of the test deterministic, as the maintainers recommend.
A release manager at a phone accessory maker
Smoke-test a new build across apps before it goes out
Queue plain-language checks in the web console in Pro mode, which plans milestones and verifies each one; at 15 to 40 seconds a step, expect a long checklist to take an afternoon, not minutes.
Before you commit
- Setup
- Clone the repo and run ./start.sh (or .\start.bat), which installs ADB, scrcpy, FFmpeg and Python dependencies with uv; then add at least one model API key to .env and connect a phone with USB debugging or an emulator.
- Integrations
- MCP for Claude Code, Codex, Cursor, Windsurf, VS Code, Cline, Antigravity and OpenClaw; Python SDK (artemis-client); models from Gemini, OpenAI, Anthropic, OpenRouter and xAI; optional Google Cloud Vision OCR.
- Data and privacy
- Apache-2.0 and self-hosted. Screenshots and prompts go to whichever model provider you configure; the on-device helper listens only on the phone, per the README.
- Cost at scale
- Nothing for the software; model spend grows with steps, and Pro's planning and checking adds calls per step. The maintainers advise turning explorations into ordinary test code so repeat runs need no model.
- Limits
- Android only; no tagged releases; 183 open issues and pull requests; traces are not compiled into reusable tests; accuracy on custom Canvas, Compose or Flutter screens depends on the visual model.
What’s good
- MCP server and rules file install into most coding agents in one step
- Web console with live mirroring, replays and reasoning traces
- Python SDK with typed results for pytest and CI
- Works with Gemini, Claude, GPT, OpenRouter or xAI models
What’s not
- Android only; iOS is a roadmap item
- 99%+ AndroidWorld score is self-reported
- Every step is a paid model call, slow in Pro mode
- Installs an accessibility helper on the device by default
Best for
Android teams who want their coding agent to reproduce bugs and explore new builds on a real device before anyone writes the regression test.
ARTEMIS alternatives
ARTEMIS: common questions
- Is ARTEMIS worth it?
- ARTEMIS is the free way to let a coding agent work a real Android phone, provided you treat it as an explorer and not as your regression suite. We rate it 4.1 out of 5.
- How much does ARTEMIS cost?
- Free under Apache-2.0. You pay for the vision-language model calls (Gemini, OpenAI, Anthropic, OpenRouter or xAI), billed per step by your provider, plus an optional Google Cloud Vision OCR key.
- What are the downsides of ARTEMIS?
- Android only; iOS is a roadmap item; 99%+ AndroidWorld score is self-reported; every step is a paid model call, slow in Pro mode; installs an accessibility helper on the device by default.
- Who is ARTEMIS best for?
- Android teams who want their coding agent to reproduce bugs and explore new builds on a real device before anyone writes the regression test.
- How much does ARTEMIS cost for a team or in production?
- Nothing for the software; model spend grows with steps, and Pro's planning and checking adds calls per step. The maintainers advise turning explorations into ordinary test code so repeat runs need no model.
- Is ARTEMIS safe for company data?
- Apache-2.0 and self-hosted. Screenshots and prompts go to whichever model provider you configure; the on-device helper listens only on the phone, per the README.
- What are the best ARTEMIS alternatives?
- The closest alternatives we have reviewed are Stagehand.
Answers checked Nov 3, 2026.
Make ARTEMIS? Show this rating on your site
