Step 21 of 22
Observability & tracing
See every prompt, tool call, cost and latency in production. When an agent misbehaves you need the trace. Observability is also how you find which costs to cut.
How to choose
- Self-hosted and open source: keep traces in your own infrastructure.
- Already on LangChain or LangGraph: the hosted tracer built for them.
- Cost dashboards per user or feature: a gateway that logs spend.
What to look for
The questions that separate the options, in order of how much they matter.
01
OpenTelemetry in and out
Instrumentation is the part that is expensive to change, so emit standard OpenTelemetry spans and keep the backend replaceable. Check whether a tool accepts OpenTelemetry (Langfuse, Opik, LangSmith, Braintrust and Weave all do), whether it is built on it (Arize Phoenix with OpenInference), and which LLM attribute conventions it reads best: OpenInference and OpenLLMetry's conventions both exist in the OpenTelemetry ecosystem.
02
Self-hosted or SaaS, and what the licence allows
Traces contain prompts, documents and answers, often with customer data. Langfuse (MIT core with commercial ee/ folders) and Opik (Apache 2.0 throughout) can be self-hosted free; Phoenix can too, but its Elastic License 2.0 forbids offering it as a hosted service to others. LangSmith, Braintrust and Weave reserve self-hosting for Enterprise contracts. Self-hosting means operating databases such as ClickHouse and Postgres, so price in the operations time.
03
Retention and PII controls
Free cloud tiers keep data for 14 to 60 days; longer retention is a paid extra almost everywhere, and LangSmith cut extended-trace retention to 180 days in September 2026. If prompts contain personal data, check for client-side redaction before upload (Braintrust's SDKs support it), server-side masking (paid on Langfuse) and per-project retention policies, and make sure your data-processing agreement covers the vendor.
04
Cost tracking per user and feature
Observability is how most teams find which costs to cut. Check that the tool records token counts and cost per span, prices models correctly including cached tokens, and can group spend by user, feature or tag. If you already route calls through a gateway, its spend logs may cover this without a separate tracer.
05
How you are billed
Pricing units differ and make quotes hard to compare: Langfuse bills units, Opik and Arize bill spans, LangSmith bills traces plus seats, Braintrust bills gigabytes processed and scores, and Weave bills ingestion per megabyte above a small allowance. Estimate a realistic month of traffic, including large prompts and tool outputs, in each vendor's own terms before choosing.
06
Evals and prompt management in the same place
The best feedback loop turns a bad production trace into a dataset entry and a regression test. Langfuse, Opik, Phoenix, LangSmith and Braintrust all combine tracing with datasets, LLM-as-judge scoring and prompt versioning; OpenLLMetry is instrumentation only and needs a backend for all of that.
07
Agent-aware trace views
Agent traces are deep trees of model calls, tool calls and sub-agents. Look for views that show sessions and turns, not just flat spans, and native integrations for your framework: LangSmith for LangGraph, Weave's one-line patches for the OpenAI Agents SDK, Claude Agent SDK and Google ADK, and broad framework coverage in Langfuse and Phoenix.
08
Ownership and roadmap risk
Almost every vendor here changed hands recently: ClickHouse bought Langfuse, Dynatrace bought Arize, ServiceNow bought Traceloop, CoreWeave owns Weights & Biases and calls Agent Lens Weave's successor, and Helicone went into maintenance mode after Mintlify acquired it. Standard instrumentation is the hedge.
How the options differ
There are three kinds of tool here. Open-source platforms (Langfuse, Opik, Arize Phoenix) store and display traces, run evaluations and manage prompts, and can be self-hosted free or bought as a cloud service; they differ mainly in licence, ecosystem size and pricing unit. Hosted platforms (LangSmith, Braintrust, W&B Weave) offer the same core with a vendor-run service, deeper ties to a framework or workflow, and self-hosting only on Enterprise plans. Instrumentation libraries (OpenLLMetry) only produce OpenTelemetry spans and send them to whatever backend you choose, including general APM tools such as Datadog or Grafana.
General error and infrastructure monitors (Sentry, Grafana) complement these tools but do not replace an LLM-aware trace view. Helicone, once a popular proxy-based option, has been in maintenance mode since March 2026 and is not recommended for new projects.
7 tools, compared
Ranked by our rating. Open any review for sources, prices and the full verdict.
| Tool | Rating | Our verdict | Best for | Price |
|---|---|---|---|---|
| Langfuserepo | 4.6/5 | Framework-neutral LLM tracing you can own outright: the free self-hosted build covers nearly everything, and the cloud is priced down to the unit. | Teams that want framework-neutral tracing and evaluation they can self-host for free or buy as a reasonably priced cloud service. | Free and open source under MIT, except ee/ folders under a commercial licence. Cloud: Hobby free (50k units/month); Core $29/month; Pro $199/month (Teams add-on +$300); Enterprise $2,499/month; overage $8 per 100k units, falling to $6 above 50M. |
| Braintrusttool | 4.3/5 | Braintrust is built for teams whose hardest question is whether the last change made the product better, and it charges by data rather than by head, which flatters big teams and punishes chatty agents. | Product teams who want evaluation-driven development with hosted tracing and are happy to pay for a managed service. | Starter free (1 GB data, 10k scores, $10 model credits, 14-day retention, unlimited users; then $4/GB and $2.50 per 1,000 scores). Pro $249/month (5 GB, 50k scores, $100 model credits, 30-day retention; then $3/GB and $1.50 per 1,000 scores). Enterprise custom, with SAML, BAA and self-hosted data plane. Qualifying startups get Pro free for 6 to 12 months. |
| Arize Phoenixrepo | 4.2/5 | Phoenix has the strongest evaluation loop of the open tracing tools, wrapped in a licence and an owner that legal teams will want to discuss. | Engineers and data scientists who want standards-based tracing with a strong evaluation loop, starting on a laptop. | Free to self-host under the Elastic License 2.0 (source-available; no hosting it as a service for others). Managed Arize AX: Free (25k spans/month, 15-day retention, 1 GB); Pro $50/month (50k spans, 30 days, 10 GB); Enterprise custom. |
| LangSmithtool | 4.2/5 | Deep, mature tracing and evaluation for LangGraph agents, sold per seat at a price that rewards small teams with big budgets. | Teams building production agents on LangChain or LangGraph who want hosted tracing, evaluation and deployment from one vendor. | Developer free (1 seat, 5,000 base traces/month). Plus $39/seat/month (10,000 base traces, then pay-as-you-go; one small serverless deployment). Enterprise custom, with hybrid or self-hosted options. Base traces kept 14 days, extended traces 180 days. |
| Opikrepo | 4.1/5 | Opik gives away the whole server under Apache 2.0 and sells the cheapest hosted plan, but the self-hosted build has no user accounts and phones home by default. | Teams that want a permissively licensed, self-hostable tracing and evaluation platform, or a low-cost hosted one. | Free and open source under Apache 2.0. Cloud: Free (25k spans/month, 60-day retention); Pro $19/month (100k spans, then $5 per 100k; 400-day retention add-on $29 per 100k spans); Enterprise custom. |
| OpenLLMetryrepo | 4.1/5 | Instrument LLM calls once in standard OpenTelemetry and keep your choice of backend, after first changing two defaults that send full prompts to Traceloop's cloud. | Teams standardised on OpenTelemetry who want LLM traces in their existing observability stack without a vendor SDK. | Free and open source under Apache 2.0. Traceloop's hosted platform, now owned by ServiceNow: Free up to 50,000 spans a month with 24-hour retention; Enterprise (SOC 2, on-prem) by quote. |
| W&B Weaverepo | 3.6/5 | Worth turning on if your models already live in Weights & Biases; at $0.10 a megabyte, a hard sell for anyone starting fresh. | Teams already using Weights & Biases for model training who want LLM traces and evaluations in the same place. | SDK free under Apache 2.0. Hosted: Free (1 GB Weave ingestion/month); Pro from $60/month (1.5 GB, then $0.10/MB); Enterprise custom ($0.20/MB overage listed), with dedicated deployment. |
Side by side
The specifications that matter for this step. Checked 2026-10-23; confirm on the vendor’s site before you commit.
| Tool | Type | Licence | Self-hosting | OpenTelemetry | Free-tier retention |
|---|---|---|---|---|---|
| Langfuse | Tracing, evals, prompts | MIT core; ee/ commercial | Free, near-complete | Accepts OTel | 30 days (Hobby) |
| Braintrust | Hosted evals and tracing | Proprietary (autoevals MIT) | Enterprise only (data plane) | Accepts OTel | 14 days |
| Arize Phoenix | Tracing, evals, prompts | Elastic License 2.0 | Free; no hosting for others | Built on OTel/OpenInference | 15 days (Arize AX Free) |
| LangSmith | Hosted tracing, evals, deploy | Proprietary | Enterprise only | Accepts OTel | 14 days (base traces) |
| Opik | Tracing, evals, prompts | Apache 2.0 | Free, same codebase as cloud | Accepts OTel | 60 days |
| OpenLLMetry | Instrumentation library only | Apache 2.0 | Uses your own backend | Emits OTel spans | Depends on backend |
| W&B Weave | Tracing and evals SDK, hosted | Apache 2.0 SDK | Enterprise dedicated only | OTLP endpoint | Not published |
Connect it to your agent
MCP servers that let assistants like Claude, ChatGPT and Cursor use these tools directly.
Where this stage fits
- Chat with your documents: we would start with Langfuse.
- Customer support agent: we would start with LangSmith.
- Coding agent: we would start with Langfuse.
- Research & knowledge graph: we would start with Arize Phoenix.
- Assistant with long-term memory: we would start with Langfuse.
- Voice agent: we would start with Langfuse.
- Document processing: we would start with Opik.
- Your own fine-tuned model: we would start with Langfuse.
- Workflow automation: we would start with Langfuse.