What it is
Ragas is a Python library for evaluating retrieval-augmented generation (RAG) apps, where a model answers from documents it retrieves. Development has effectively stopped: the last release, v0.4.3, came out on 13 January 2026, the last commit landed on 24 February 2026, and community pull requests opened since then have not been merged. The GitHub organisation moved from explodinggradients to vibrantlabsai, and Vibrant Labs now describes itself as a research lab selling reinforcement-learning data for agents.
The library offers metrics such as faithfulness, answer relevancy and context precision, most of which ask a judge model to score each sample, plus custom LLM-judged metrics and synthetic test-set generation. It integrates with LangChain, LlamaIndex, Haystack and Amazon Bedrock models.
It is still heavily used: PyPI recorded about 1.05 million downloads in the month to 23 October 2026.
Our review
Ragas is now a library to keep running rather than one to adopt. Its definitions of faithfulness and context precision are cited across tutorials and rival tools, and it still draws about a million PyPI downloads a month, but nobody is steering it. The last commit was on 24 February 2026, roughly 250 pull requests sit open, and an issue filed on 9 October 2026 titled "This project is not maintained anymore" has no reply.
The open pull requests show what users are hitting. Several fix answer relevancy blocking the async event loop with synchronous embedding calls, which stalls unrelated tasks during evaluate(); others fix few-shot thresholds that were silently ignored and Bedrock reasoning model IDs that were not mapped. Each is a patch you would have to apply from a fork.
The cost model matters at volume. Most Ragas metrics are judged by an LLM, so every sample triggers one or more model calls, and scores can shift between runs and between judge models. A team scoring thousands of samples a night is paying API fees and absorbing that variance, not running a free test suite.
Telemetry is the detail most readers miss. Ragas sends anonymous usage events to an explodinggradients.com endpoint by default, and the opt-out is an environment variable, RAGAS_DO_NOT_TRACK=true. In a regulated environment that should be set before the first run, whatever else you decide about the library.
Where it earns its place
An ML engineer on a five-person team with a Ragas suite already in CI
Keep nightly RAG regression scores comparable while deciding what to migrate to
Pin ragas==0.4.3, set RAGAS_DO_NOT_TRACK=true and fix the judge model so scores stay comparable. Cost is the judge-model API calls per sample; the risk is carrying fixes for bugs such as the event-loop blocking in answer relevancy yourself.
A researcher writing an evaluation section for a RAG paper
Report faithfulness and context precision under definitions reviewers recognise
Ragas metric definitions are widely cited, so using them for a one-off study needs little justification. Record the version and judge model, since LLM-judged scores move with both.
A head of AI at a 200-person insurer choosing an evaluation stack
Pick an eval library the team can rely on for the next two years
Ragas costs nothing to license, but with no maintainer every bug fix and model update falls to your team. DeepEval or promptfoo carry the same RAG metrics with active releases.
Before you commit
- Setup
- pip install ragas on Python 3.9 or newer, plus an API key for a judge model; ragas quickstart rag_eval scaffolds an example project in a few minutes.
- Integrations
- LangChain, LlamaIndex and Haystack wrappers, OpenAI, Anthropic, Google and Bedrock models through llm_factory, and the AG-UI message format.
- Data and privacy
- Apache 2.0. Anonymous usage telemetry is on by default and sent to t.explodinggradients.com; set RAGAS_DO_NOT_TRACK=true to stop it. Your samples go to whichever judge-model API you configure. No published security advisories.
- Cost at scale
- The library is free and there is no paid edition. Real cost is judge-model tokens: LLM-judged metrics make at least one model call per sample, so a large nightly suite is an API bill.
- Limits
- 634 open issues and pull requests, none merged since February 2026; known bugs include synchronous embedding calls blocking the event loop in answer relevancy. Only the rag_eval quickstart template exists; agent templates are listed as coming soon.
What's good
- Widely cited definitions of faithfulness and context precision
- Synthetic test-set generation from your documents
- Integrations with LangChain, LlamaIndex and Haystack
- Apache 2.0 licence
What's not
- No commits since 24 February 2026; about 250 unmerged pull requests
- Usage telemetry on by default
- LLM-judged scores vary between runs and judge models
- API changes between versions broke many online examples
Best for
Teams with an existing Ragas suite who can pin a version and carry their own fixes.
Alternatives
Ragas: common questions
- Is Ragas worth it?
- Its metric names became the vocabulary of RAG evaluation, but the code has been left without a maintainer. We rate it 3.4 out of 5.
- How much does Ragas cost?
- Free and open source under the Apache 2.0 licence. You pay for the judge-model API calls each metric makes.
- What are the downsides of Ragas?
- No commits since 24 February 2026; about 250 unmerged pull requests; usage telemetry on by default; lLM-judged scores vary between runs and judge models; aPI changes between versions broke many online examples.
- Who is Ragas best for?
- Teams with an existing Ragas suite who can pin a version and carry their own fixes.
- How much does Ragas cost for a team or in production?
- The library is free and there is no paid edition. Real cost is judge-model tokens: LLM-judged metrics make at least one model call per sample, so a large nightly suite is an API bill.
- Is Ragas safe for company data?
- Apache 2.0. Anonymous usage telemetry is on by default and sent to t.explodinggradients.com; set RAGAS_DO_NOT_TRACK=true to stop it. Your samples go to whichever judge-model API you configure. No published security advisories.
- What are the best Ragas alternatives?
- The closest alternatives we have reviewed are DeepEval and promptfoo.
Answers checked Oct 23, 2026.