What it is
OpenLLMetry is a set of OpenTelemetry extensions that record LLM calls as standard spans. It instruments model providers including OpenAI and Azure OpenAI, Anthropic, AWS Bedrock and SageMaker, Google Gemini and Vertex AI, Cohere, Mistral, Groq, Ollama and Hugging Face, plus vector databases such as Chroma, Pinecone, Qdrant, Milvus and Weaviate, agent frameworks such as LangGraph, CrewAI and the OpenAI Agents SDK, and MCP. The Traceloop SDK sets it up in one line; teams already using OpenTelemetry can add the individual instrumentations instead.
It is not a backend. Traces go to any OpenTelemetry destination, and the README lists 24 tested ones, including Datadog, Grafana, Honeycomb, New Relic, Dynatrace, Splunk, Sentry, Braintrust and a plain OpenTelemetry Collector. The project says its semantic conventions for LLM spans are now part of OpenTelemetry. A separate repository covers JavaScript and TypeScript.
It is built by Traceloop and licensed under Apache 2.0; version 0.62.4 was released on 29 September 2026. ServiceNow announced its acquisition of Traceloop in March 2026 and described it as completed on 5 May, folding the hosted product into its AI Control Tower.
Our review
OpenLLMetry solves the lock-in problem at the right layer. Instead of adopting a vendor SDK, you emit standard OpenTelemetry spans and route them to the observability tool you already pay for, or to Langfuse, Braintrust or Phoenix, and change your mind later without touching application code. It has about 7,500 GitHub stars, but the star count understates its reach: frameworks such as Haystack and LlamaIndex document it as their tracing route.
The defaults deserve a closer look than the one-line setup suggests. Called with no exporter, Traceloop.init() sends traces to api.traceloop.com and generates an API key automatically, and the privacy docs say prompts, completions and embeddings are written to span attributes by default. A team that installs it to feed Datadog can therefore ship customer prompts to a third party on day one. Setting TRACELOOP_BASE_URL or a custom exporter, and TRACELOOP_TRACE_CONTENT=false where content is sensitive, fixes both. The SDK itself stopped collecting telemetry from v0.49.2.
On its own it stores nothing and shows nothing: no trace viewer, no cost dashboard, no evals. LLM-aware backends such as Langfuse or Phoenix make far better use of the spans than a general APM tool, which may show a prompt as an opaque attribute. Arize's competing OpenInference conventions coexist, and backends vary in which they read best.
Ownership is the long-term question. ServiceNow bought Traceloop for its AI Control Tower, and its hosted free tier keeps only 24 hours of data, which reads as a funnel to Enterprise. The instrumentations are Apache 2.0 and portable, so the realistic risk is slower maintenance as providers change their APIs, not a licence change.
Where it earns its place
An SRE team at a retailer already paying for Datadog
See LLM latency, errors and token counts next to the rest of the stack
pip install traceloop-sdk, call Traceloop.init() with the OTLP exporter pointed at the Datadog agent, and set TRACELOOP_TRACE_CONTENT=false if prompts hold customer data. No new vendor bill; Datadog charges for the extra spans at its usual rates.
A platform engineer standardising tracing across five LLM services
Instrument once and keep the option to switch backends
Add only the opentelemetry-instrumentation-openai and -anthropic packages to an existing OpenTelemetry setup, export to a Collector, and fan out to Langfuse for LLM debugging and Grafana for dashboards.
A head of engineering at a 20-person start-up
Get LLM observability without paying for a platform yet
Traceloop's hosted Free tier covers 50,000 spans a month, each LLM or vector call counting as one, with 24-hour retention; a self-hosted Langfuse or Phoenix behind OpenLLMetry keeps longer history for the cost of a small server.
Before you commit
- Setup
- pip install traceloop-sdk and one Traceloop.init() line, or add individual opentelemetry-instrumentation-* packages to an existing OpenTelemetry setup; traces appear within minutes once an exporter is set.
- Integrations
- 16 model providers, 7 vector databases, frameworks including LangChain, LangGraph, LlamaIndex, CrewAI, Agno and OpenAI Agents, MCP, and 24 tested destinations; a JavaScript port exists.
- Data and privacy
- Apache 2.0. Prompts, completions and embeddings are captured by default (TRACELOOP_TRACE_CONTENT=false disables), and the SDK exports to api.traceloop.com unless you configure another endpoint. No SDK telemetry since v0.49.2; no published GitHub security advisories.
- Cost at scale
- The library is free; cost is the backend. Traceloop Free covers 50,000 spans a month with 24-hour retention, Enterprise is quoted; with Datadog, Honeycomb or a self-hosted Langfuse you pay their span or hosting rates.
- Limits
- About 800 open issues and pull requests, instrumentation breakage when provider SDKs change, and span attributes that general APM tools display poorly compared with LLM-specific backends.
What's good
- Standard OpenTelemetry spans, no vendor lock-in
- Instruments most providers, vector databases and agent frameworks
- 24 tested backends
- No SDK telemetry since v0.49.2
What's not
- Defaults send traces, with prompt content, to Traceloop cloud
- Not a backend; needs a destination
- Competing semantic conventions with OpenInference
- Owner now ServiceNow
Best for
Teams standardised on OpenTelemetry who want LLM traces in their existing observability stack without a vendor SDK.
Alternatives
OpenLLMetry: common questions
- Is OpenLLMetry worth it?
- Instrument LLM calls once in standard OpenTelemetry and keep your choice of backend, after first changing two defaults that send full prompts to Traceloop's cloud. We rate it 4.1 out of 5.
- How much does OpenLLMetry cost?
- Free and open source under Apache 2.0. Traceloop's hosted platform, now owned by ServiceNow: Free up to 50,000 spans a month with 24-hour retention; Enterprise (SOC 2, on-prem) by quote.
- What are the downsides of OpenLLMetry?
- Defaults send traces, with prompt content, to Traceloop cloud; not a backend; needs a destination; competing semantic conventions with OpenInference; owner now ServiceNow.
- Who is OpenLLMetry best for?
- Teams standardised on OpenTelemetry who want LLM traces in their existing observability stack without a vendor SDK.
- How much does OpenLLMetry cost for a team or in production?
- The library is free; cost is the backend. Traceloop Free covers 50,000 spans a month with 24-hour retention, Enterprise is quoted; with Datadog, Honeycomb or a self-hosted Langfuse you pay their span or hosting rates.
- Is OpenLLMetry safe for company data?
- Apache 2.0. Prompts, completions and embeddings are captured by default (TRACELOOP_TRACE_CONTENT=false disables), and the SDK exports to api.traceloop.com unless you configure another endpoint. No SDK telemetry since v0.49.2; no published GitHub security advisories.
- What are the best OpenLLMetry alternatives?
- The closest alternatives we have reviewed are Langfuse, Arize Phoenix and Opik.
Answers checked Oct 23, 2026.
Make OpenLLMetry? Show this rating on your site
