What it is
DeepTeam is an open-source Python framework from Confident AI, the company behind DeepEval, for red-teaming LLM systems. It simulates attacks such as jailbreaks, prompt injection, encoding obfuscation and multi-turn exploitation, then uses LLM-as-judge metrics to give each test a pass or fail with reasoning.
The README lists more than 50 vulnerabilities and more than 20 attack methods. Beyond PII leakage, prompt leakage, bias and toxicity, it covers application-security classes such as broken function- and object-level authorisation, SQL and shell injection and server-side request forgery, and agent-specific ones such as excessive agency, indirect instructions in retrieved content, tool metadata poisoning and inter-agent spoofing. Presets map OWASP Top 10 for LLMs 2025, OWASP Top 10 for Agents 2026, NIST AI RMF, MITRE ATLAS, BeaverTails and Aegis to the right tests, and it ships seven runtime guardrails.
It runs locally from Python or a YAML-driven CLI and is Apache 2.0-licensed. Version 1.0.10 was published to PyPI on 16 October 2026; the GitHub releases page stops at v1.0.9. Confident AI's hosted platform sells storage and reporting, with red teaming an Enterprise add-on.
Our review
DeepTeam's strength is coverage of the risks that matter once a model has tools. Its vulnerability list reads like an application-security checklist translated for agents: authorisation bypass, SQL and shell injection, SSRF, tool metadata poisoning and instructions hidden in retrieved documents. Six framework presets, including the 2026 OWASP list for agents, give a team a defensible baseline in one call when a security review asks what was tested.
The weakness is the judge. An LLM generates the attacks and another decides pass or fail, so results move with the judge model, every run costs tokens on your OpenAI key or whichever model you configure, and both false alarms and misses are expected. garak's rule-based detectors and PyRIT's scorer options give more control over that step.
Two defaults deserve attention. Telemetry is on unless you set DEEPTEAM_TELEMETRY_OPT_OUT=YES (and DEEPEVAL_TELEMETRY_OPT_OUT=YES for the underlying library): the code initialises Sentry and PostHog and looks up your public IP address through ipify. The data-privacy page says only names of vulnerabilities run and usage counts are sent, not prompts. Second, the GitHub releases page last updated in November 2025 while PyPI moved on to 1.0.10, so changelogs are thin.
The business model is visible. The free library sends you towards Confident AI for storing results, and on its pricing page red teaming is not in the $200 Starter or $2,000 Team plans but an "Enterprise ++" module at custom prices. A team that wants scheduled assessments and shared reports without that contract will end up building its own JSON-to-dashboard plumbing, which is what promptfoo gives away in its open-source CLI.
Where it earns its place
A security engineer at a fintech shipping a support agent with refund and account tools
Show the audit team the agent was tested against OWASP before launch
Wrap the agent in an async model_callback and run red_team with the OWASP_ASI_2026 framework; export the risk assessment as JSON for the review. Budget judge-model tokens per run and have a person confirm each failure.
An ML platform team running nightly checks on a RAG chatbot
Catch regressions in prompt-injection and PII-leak resistance after prompt changes
Keep a YAML config with chosen vulnerabilities and attacks and run the CLI in CI. Set the two telemetry opt-out variables in the runner, and pin the judge model so pass rates are comparable night to night.
A CISO at a 500-person insurer comparing red-teaming options
Decide between open-source tooling and a managed red-teaming service
DeepTeam itself is free, but Confident AI sells hosted red teaming only as an Enterprise add-on with custom pricing. Price that against staff time to run the library and triage LLM-judge findings by hand.
Before you commit
- Setup
- pip install -U deepteam on Python 3.9 to 3.13, set OPENAI_API_KEY or configure another judge model, write a model_callback; a first single-vulnerability run takes minutes.
- Integrations
- Built on DeepEval; any DeepEval-supported model as attacker and judge; CLI with YAML configs; code-scanning GitHub bot in the docs; results push to Confident AI.
- Data and privacy
- Prompts go to whichever attacker and judge models you configure. Telemetry via Sentry and PostHog with a public-IP lookup is on by default; opt out with DEEPTEAM_TELEMETRY_OPT_OUT=YES and DEEPEVAL_TELEMETRY_OPT_OUT=YES. No GitHub security advisories published.
- Cost at scale
- The library is free; each run costs attacker and judge tokens that scale with vulnerabilities x attacks x turns. Confident AI: Free (2 seats, 5 test runs a week), Starter $200 a month, Team $2,000 a month with SOC 2 and SSO; red teaming is an Enterprise add-on.
- Limits
- 80 open issues and pull requests; judge variance between models; multi-turn attacks multiply token spend; GitHub releases stop at v1.0.9 from November 2025.
What's good
- Wide coverage of agent and app-security vulnerabilities
- OWASP, NIST and MITRE ATLAS presets
- Python API and YAML-driven CLI, both local
- Apache 2.0 with frequent PyPI releases
What's not
- Pass/fail depends on an LLM judge
- Telemetry on by default, including a public-IP lookup
- Hosted red-teaming module is Enterprise-only
- GitHub release notes lag PyPI by a year
Best for
Python teams building agents who want OWASP-mapped red-team coverage they can run locally.
Alternatives
DeepTeam: common questions
- Is DeepTeam worth it?
- The broadest catalogue of agent and application vulnerabilities in an open red-teaming library, but every verdict rests on an LLM judge you will need to check. We rate it 3.9 out of 5.
- How much does DeepTeam cost?
- Free and open source under Apache 2.0. Confident AI's platform runs from a free tier to $200 and $2,000 a month; its red-teaming module is an Enterprise add-on at custom pricing.
- What are the downsides of DeepTeam?
- Pass/fail depends on an LLM judge; telemetry on by default, including a public-IP lookup; hosted red-teaming module is Enterprise-only; gitHub release notes lag PyPI by a year.
- Who is DeepTeam best for?
- Python teams building agents who want OWASP-mapped red-team coverage they can run locally.
- How much does DeepTeam cost for a team or in production?
- The library is free; each run costs attacker and judge tokens that scale with vulnerabilities x attacks x turns. Confident AI: Free (2 seats, 5 test runs a week), Starter $200 a month, Team $2,000 a month with SOC 2 and SSO; red teaming is an Enterprise add-on.
- Is DeepTeam safe for company data?
- Prompts go to whichever attacker and judge models you configure. Telemetry via Sentry and PostHog with a public-IP lookup is on by default; opt out with DEEPTEAM_TELEMETRY_OPT_OUT=YES and DEEPEVAL_TELEMETRY_OPT_OUT=YES. No GitHub security advisories published.
- What are the best DeepTeam alternatives?
- The closest alternatives we have reviewed are promptfoo, PyRIT and garak.
Answers checked Oct 23, 2026.