What is The AI Scientist (v1, v2 and v3)?
The AI Scientist is Sakana AI's line of systems that take a research direction and produce experiments and a written paper with little or no human help. It now spans three repositories. The original AI-Scientist (August 2024, about 14,800 stars) works inside human-written code templates for NanoGPT, 2D diffusion and grokking: an LLM proposes ideas, edits the template, runs the experiments and writes a LaTeX paper, and an LLM reviewer scores it. AI-Scientist-v2 (April 2025, about 7,400 stars) drops the templates and searches over experiments with an agentic tree search built on WecoAI's AIDE; Sakana warns it has lower success rates than v1 where a good template exists.
AI-Scientist-v3, published on 11 October 2026 and last updated on 18 October, is a new design called the Agentic Scientist. It drives existing coding agents (Claude Code, Codex, Cursor, OpenCode, Gemini CLI and Pi) through six phases: task drafting, investigation, ablations, a sealed final evaluation, post-hoc analysis and paper writing, each in its own Docker container. It has about 520 stars.
v1 and v2 moved in December 2025 from Apache 2.0 to the AI Scientist Source Code License 1.0, a Responsible AI License derivative that legally requires you to disclose AI generation in any resulting paper; neither has had a commit since 19 December 2025. v3 is Apache 2.0. None of the three publishes releases.
Is The AI Scientist (v1, v2 and v3) worth it?
The AI Scientist is more important as a research artefact than as a tool, and v3 is the first version that is plausibly a tool. v1 showed in 2024 that a frontier model could go from idea to formatted paper for under $15 inside a template; v2 removed the template and produced the paper that passed peer review at an ICLR 2025 workshop; the Nature paper of March 2026 formalised the pipeline and reported that paper quality rises with the underlying model. v3 changes the premise: instead of one model writing everything, it wraps commercial coding agents in rubric checks, parallel workers and an audit trail, and Sakana frames it as co-discovery with a human setting the question.
The headline results need their context. At the ICBINB workshop Sakana submitted three v2 papers among 43 submissions; one scored 6, 7 and 6 (average 6.33) and was withdrawn before publication as planned, two were rejected, and Sakana's own review found none good enough for the main conference and noted citation errors such as crediting LSTMs to Goodfellow (2016). An independent evaluation by Joeran Beel, Min-Yen Kan and Moritz Baumgart, published in SIGIR Forum, found that 5 of 12 proposed experiments (42%) failed on coding errors, novelty checks labelled known ideas such as micro-batching for SGD as new, papers had a median of five mostly old citations, and several manuscripts contained invented numbers. They likened the output to an unmotivated undergraduate rushing a deadline, while calling the autonomy a real milestone. v3's ALIFE 2026 paper was a co-scientist result where humans chose the question and checked every result, and its reported Doc-to-LoRA gain from 52% to 98% exact match is Sakana's own.
Safety is a design constraint, not a footnote. All three versions execute LLM-written code. v1 and v2 tell you to containerise and restrict web access yourself; v1 relies on a community-contributed Dockerfile. v3 runs each phase in Docker by default, but its agents run with full tool permissions (Headless --allow yolo), can read the API keys you mount, and can be steered by the task files and papers they read.
Costs are modest per run and add up across a search. Sakana's FAQs put v1 at typically under $15 a paper with Claude 3.5 Sonnet, and v2 at $15 to $20 for experiments plus about $5 for writing; Beel's team measured $6 to $15 and about 3.5 hours of human involvement per v1 paper. v3 logs per-phase token use and API-list-price estimates to a cost ledger and can bill a Claude or ChatGPT subscription instead of an API key; its GRPO example needs two GPUs and trains for one to two hours per evaluation. For engineers who want automated optimisation of a single scored program rather than papers, ShinkaEvolve from the same lab is the more direct fit.
Where it earns its place
An ML lab with a scored benchmark task and spare GPUs
Have coding agents explore ideas, run ablations and draft a write-up overnight
Point v3's agsci run at a task directory with an evaluator; it runs investigation, ablations and a sealed evaluation in Docker and writes a paper. Expect to verify every claim, as Sakana's own ALIFE paper was human-checked.
A researcher studying AI-generated science
Generate a corpus of machine-written papers for analysis
Run v1 on its NanoGPT, 2D diffusion or grokking templates at typically under $15 a paper. The licence requires every resulting manuscript to disclose that AI produced it.
A university course on research methods
Show students what automated research gets wrong
Pair v1 or v2 output with Beel et al.'s evaluation, which documents failed experiments, weak novelty checks and invented numbers, as a reviewing exercise.
Before you commit
- Setup
- v1/v2: conda with Python 3.11, PyTorch with CUDA on Linux and NVIDIA GPUs, LaTeX tools (texlive-full for v1), API keys; v2 install usually takes under an hour. v3: Docker plus uv sync, build the agent image, put keys in .env, then agsci run examples/circpack runs CPU-only with a Claude login.
- Integrations
- v1: OpenAI, Anthropic (direct, Bedrock, Vertex), DeepSeek, OpenRouter and Gemini, with Semantic Scholar or OpenAlex for literature search. v3: coding agents through Headless CLI, an optional OpenAI-backed reviewer phase, a browser run inspector on port 8765, and Autoillustrate for figures.
- Data and privacy
- Prompts and code go to whichever LLM provider you configure; literature queries go to Semantic Scholar or OpenAlex. v3 forwards only the selected agent's keys into its containers, and Sakana warns that agent logs can echo document contents. The v1/v2 licence bars surveillance, some health and criminal-prediction uses, and undisclosed AI papers.
- Cost at scale
- Sakana's figures: v1 typically under $15 a paper with Claude 3.5 Sonnet; v2 $15 to $20 per experiment run plus about $5 to write. A 50-idea sweep on v2 therefore runs to roughly $1,000 to $1,250 in API fees before GPU time.
- Limits
- Computational experiments only. v2 succeeds less often than v1 when a strong template exists. v3 task drafting from a bare brief is experimental, literature search happens only at writing time, and baselines are mostly variants of its own program.
What's good
- Three generations of a well-documented automated research pipeline
- v3 drives Claude Code, Codex, Cursor, OpenCode, Gemini CLI and Pi interchangeably
- v3 runs every phase in Docker by default with a full audit trail
- Low cost per paper on Sakana's figures ($15 to $25 for v2)
- Candid README warnings about running LLM-written code
What's not
- Generated papers contain citation errors, failed experiments and invented numbers (independent evaluation)
- v1 and v2 need Linux, NVIDIA GPUs and texlive; no commits since December 2025
- v3 agents run with full permissions and can be steered by inputs
- v1 and v2 licence restricts uses and requires AI disclosure in papers
- Results limited to computational ML experiments
Best for
ML research groups studying or using automated experimentation who can sandbox agent runs and will review every result before it is published.
The AI Scientist (v1, v2 and v3) alternatives
The AI Scientist (v1, v2 and v3): common questions
- Is The AI Scientist (v1, v2 and v3) worth it?
- The reference project for automated ML research and now a usable harness for coding agents, but its papers still need a human to catch errors, and every run executes code an LLM wrote. We rate it 3.9 out of 5.
- How much does The AI Scientist (v1, v2 and v3) cost?
- Free. v1 and v2 under the AI Scientist Source Code License 1.0 (use restrictions and mandatory AI disclosure); v3 under Apache 2.0. You pay for LLM API calls or subscriptions and GPUs.
- What are the downsides of The AI Scientist (v1, v2 and v3)?
- Generated papers contain citation errors, failed experiments and invented numbers (independent evaluation); v1 and v2 need Linux, NVIDIA GPUs and texlive; no commits since December 2025; v3 agents run with full permissions and can be steered by inputs; v1 and v2 licence restricts uses and requires AI disclosure in papers; results limited to computational ML experiments.
- Who is The AI Scientist (v1, v2 and v3) best for?
- ML research groups studying or using automated experimentation who can sandbox agent runs and will review every result before it is published.
- How much does The AI Scientist (v1, v2 and v3) cost for a team or in production?
- Sakana's figures: v1 typically under $15 a paper with Claude 3.5 Sonnet; v2 $15 to $20 per experiment run plus about $5 to write. A 50-idea sweep on v2 therefore runs to roughly $1,000 to $1,250 in API fees before GPU time.
- Is The AI Scientist (v1, v2 and v3) safe for company data?
- Prompts and code go to whichever LLM provider you configure; literature queries go to Semantic Scholar or OpenAlex. v3 forwards only the selected agent's keys into its containers, and Sakana warns that agent logs can echo document contents. The v1/v2 licence bars surveillance, some health and criminal-prediction uses, and undisclosed AI papers.
- What are the best The AI Scientist (v1, v2 and v3) alternatives?
- The closest alternatives we have reviewed are ShinkaEvolve, OpenHands, TreeQuest and mini-swe-agent.
Answers checked Oct 26, 2026.