What is Giskard?
Giskard is an open-source Python library for testing LLM applications and agents. Version 3 is a rewrite split into small packages: giskard-checks runs scenario tests, in which a target (any sync or async function) is sent a sequence of inputs and the trace is checked with string matches, regexes, semantic similarity or LLM judges such as Groundedness and Conformity; giskard-scan adds a vulnerability scan that generates adversarial suites from a plain-language description of your agent, and a quality scan that builds RAG test questions from a knowledge base.
The scan covers prompt injection, harmful content, stereotypes and misinformation, mapped to the OWASP LLM Top 10, and ships a dataset of injection payloads. It needs Python 3.12 or later and an LLM provider key for the judges and generators; the default model is openai/gpt-4o-mini.
The repository, Giskard-AI/giskard-oss, has about 5,900 stars and is Apache-2.0. The company sells an Enterprise platform for continuous red teaming on top, with on-premises, private cloud and SaaS deployment and no published price.
Is Giskard worth it?
Giskard's case is breadth in one dependency. A team can write deterministic and LLM-judged checks for a multi-turn conversation, generate a RAG question set from its own documents, and run an adversarial scan for prompt injection and harmful output, all from the same Python package. Promptfoo covers similar ground from a YAML-first CLI; DeepEval and Ragas each cover one slice.
The v3 design is sensible. Everything is a target, a scenario, a check and a suite, so a raw model call, a LangChain agent and a black-box HTTP endpoint are tested the same way, and only the packages you install pull in dependencies. The vulnerability scan writes its suite from a description of what the agent should and should not do, which is quicker than hand-writing attack prompts, and custom generators can be registered alongside the built-in ones.
The catch is the rewrite. v2 is no longer actively maintained, and its automatic scan for tabular and classic ML models, with the Hub integration, is not planned for v3. Projects on giskard.Model and giskard.Dataset face a migration rather than an upgrade. Python 3.12 is the floor, which excludes many older production images.
Two defaults deserve attention. Every LLM judge and scan generator calls a paid provider, gpt-4o-mini unless configured, so a large scan has a token bill and sends your test prompts to that provider. Anonymous telemetry is on by default; the README says no prompts or outputs are sent, and DO_NOT_TRACK=1 turns it off.
For a team whose question is "is this agent safe and grounded enough to ship", Giskard is a strong open-source starting point; teams needing CI dashboards and history will still pair it with a tracing platform.
Where it earns its place
An ML engineer shipping a customer-support agent at a 60-person insurer
Check the agent cannot be talked into leaking policy data before launch
Describe the agent's role and forbidden actions, run the vulnerability scan, and keep the generated prompt-injection scenarios as a regression suite. Cost is the LLM tokens the generators and judges use.
A platform team running a RAG assistant over internal documentation
Measure whether answers stay grounded after re-indexing
Point quality_scan at the knowledge base to generate questions, then run Groundedness checks in CI on every index rebuild; a drop shows up as failing scenarios rather than user complaints.
Before you commit
- Setup
- pip install giskard for checks, or giskard[scan] for the scans, plus a provider extra such as giskard[openai] and its API key; Python 3.12 or later.
- Integrations
- Wraps any sync or async callable, so it works with LangChain, LlamaIndex, raw SDK calls or an HTTP endpoint; LLM calls route to any provider through giskard-llm; runs under pytest or in notebooks.
- Data and privacy
- Runs locally; test prompts go to whichever LLM provider you configure for judges and generators. Aggregated telemetry is on by default without prompts or outputs, disabled with DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1.
- Cost at scale
- The library is free; the running cost is judge and generator tokens. The Enterprise platform with continuous red teaming and SSO has no published price.
- Limits
- v2's tabular and ML model scan is not planned for v3; the pricing page lists 50+ adversarial probes for the Enterprise platform, not the library.
What’s good
- Scenario tests, RAG evaluation and red-team scan in one Apache-2.0 library
- Adversarial suites generated from a plain-language agent description
- Mapped to the OWASP LLM Top 10, with a built-in injection payload set
- Modular packages keep dependencies small
What’s not
- v3 drops the tabular and ML model scan; v2 no longer actively maintained
- Requires Python 3.12 or later
- Judges and scans call a paid LLM provider by default
- Telemetry on unless you opt out
Best for
Python teams that want agent scenario tests and a pre-launch red-team scan from one open-source library.
Giskard alternatives
Giskard: common questions
- Is Giskard worth it?
- Giskard puts multi-turn agent tests and an OWASP-mapped red-team scan in one Apache-2.0 library, but v3 is young and drops the tabular-model scan v2 users relied on. We rate it 4.0 out of 5.
- How much does Giskard cost?
- Library free and open source (Apache-2.0). Enterprise platform (continuous red teaming, SSO, RBAC, on-premises or SaaS) priced on request.
- What are the downsides of Giskard?
- v3 drops the tabular and ML model scan; v2 no longer actively maintained; requires Python 3.12 or later; judges and scans call a paid LLM provider by default; telemetry on unless you opt out.
- Who is Giskard best for?
- Python teams that want agent scenario tests and a pre-launch red-team scan from one open-source library.
- How much does Giskard cost for a team or in production?
- The library is free; the running cost is judge and generator tokens. The Enterprise platform with continuous red teaming and SSO has no published price.
- Is Giskard safe for company data?
- Runs locally; test prompts go to whichever LLM provider you configure for judges and generators. Aggregated telemetry is on by default without prompts or outputs, disabled with DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1.
- What are the best Giskard alternatives?
- The closest alternatives we have reviewed are promptfoo, DeepEval, DeepTeam and garak.
Answers checked Nov 6, 2026.
Make Giskard? Show this rating on your site
