Langfuse · LangSmith · Braintrust · Arize Phoenix · Helicone · Weights & Biases (Weave) · WhyLabs · Datadog LLM Observability · New Relic AI Monitoring · Traceloop / OpenLLMetry.
One question: which one is right for your stage?

1. Langfuse Series A · open-source · self-host or hosted cloud · fastest-growing OSS pick · best balance of features + cost

The open-source LLM observability leader and best balance of features + cost in the category — the substrate-of-choice when 'I want full tracing + evals + prompt management + cost tracking and I want the option to self-host' is the bar. Langfuse ships traces, evals, prompt management, and cost monitoring under MIT license — the most complete OSS feature set in the category. Self-host on Docker / Kubernetes for full data control, or use Langfuse Cloud for zero-ops. SDK coverage spans Python, JS/TS, Java, Go, plus OpenTelemetry compatibility. AI-baked-in (Langfuse was built specifically for LLM tracing from day one — never a general-purpose APM retrofitting LLM modules). The default OSS-or-hosted observability substrate when 'two trillion-dollar companies wired by SideGuy' includes a monitoring layer that won't lock you in.

2. LangSmith LangChain Inc. · LangChain-native · official tracing for LangChain/LangGraph · hosted SaaS (self-host enterprise tier)

The LangChain-native observability layer — the right pick when your team is already deeply committed to LangChain or LangGraph and you want first-party tracing for those frameworks. LangSmith is built by LangChain Inc. — every LangChain chain, every LangGraph agent step, every callback emits structured traces into LangSmith out of the box with zero glue code. Strong eval framework with LangChain-native datasets + grading. Hosted SaaS with enterprise self-host tier emerging. AI-baked-in (built specifically for LangChain LLM workloads). The procurement-defensible pick for LangChain shops; less compelling for teams that don't run LangChain.

3. Braintrust Series A · evals-first architecture · best for production evals + regression testing · dev-favorite

The evals-first LLM observability platform — the right pick when 'I want a real eval discipline with offline test suites, CI integration, A/B model comparison, and golden datasets' is the deciding axis. Braintrust built the deepest eval framework in the category: offline eval suites runnable in CI, online evals on production traffic, autoeval scoring with LLM-as-judge, A/B model + prompt comparison with statistical significance, dataset versioning + golden-set management. Tracing is solid but secondary — Braintrust's lane is evals as a first-class engineering discipline. AI-baked-in. Loved by teams shipping AI features that need to NOT regress between model versions or prompt changes.

4. Arize Phoenix Open-source (Apache 2.0) · evals + tracing · OpenTelemetry-native · multi-framework support · self-host or hosted

The open-source Apache 2.0 LLM observability platform with strong evals + tracing + multi-framework support — the right pick for teams that want OSS inspectability AND eval depth without vendor lock-in. Arize Phoenix is the OSS-first sibling of Arize AI's enterprise ML observability platform — runs locally as a Python notebook companion or self-hosted in production. OpenTelemetry-native (vendor-neutral spans), multi-framework support (LangChain · LlamaIndex · OpenAI SDK · Anthropic SDK · LiteLLM · Haystack · etc.), strong eval framework with LLM-as-judge + human-in-the-loop. AI-baked-in. The OSS Apache-2.0 alternative when MIT-licensed Langfuse doesn't fit and you need eval depth Braintrust offers in hosted form.

5. Helicone Series A · proxy-based architecture · 1-line install · best for fast prototyping + cost tracking · open-source (MIT)

The proxy-based drop-in observability layer — the right pick when 'I want to wire LLM monitoring in 60 seconds with one line of code change' is the bar. Helicone runs as a proxy in front of your LLM provider (change your OpenAI base URL to https://oai.helicone.ai/v1, that's it) — captures every request + response + cost + latency without SDK instrumentation. The fastest install in the category. Open-source MIT, hosted SaaS or self-host. Strong cost tracking + caching + rate-limiting as proxy-layer features the SDK-based competitors can't match natively. AI-baked-in. Trade-off: proxy architecture means Helicone is in your hot path (latency + uptime dependency).

6. Weights & Biases (Weave) Late-stage · ML-platform-native · best for teams already on W&B for ML model tracking · hosted SaaS (self-host enterprise)

The ML-platform-native LLM observability layer from Weights & Biases — the right pick when your team is already on W&B for ML experiment tracking and you want LLM observability under the same roof. Weave (W&B's LLM observability product) ships traces, evals, prompt + dataset versioning, and online monitoring inside the W&B platform alongside ML experiment tracking. Strong for teams that have both classical ML and LLM workloads — same auth, same UI, same procurement contract. Hosted SaaS with enterprise self-host tier. AI-baked-in (W&B was built for ML observability from day one and Weave extended that to LLMs natively).

7. WhyLabs Series A · enterprise observability + drift monitoring · regulated/enterprise scale · hosted SaaS

The enterprise-scale LLM + ML observability platform with deep drift monitoring — the right pick for regulated industries (finance · healthcare · government) where data drift + model drift + performance regression are auditable concerns. WhyLabs ships LangKit for LLM monitoring (toxicity, jailbreak, PII detection, hallucination scoring) plus the broader WhyLabs platform for traditional ML drift monitoring. Strong for teams with formal MLOps + AIOps practices and compliance requirements. Hosted SaaS with strong enterprise compliance posture. AI-baked-in for the LangKit LLM module; ML-baked-in for the broader platform.

8. Datadog LLM Observability Datadog · enterprise APM-native · best for teams already on Datadog who want one pane of glass · 2024 GA

LLM observability bundled into Datadog APM — the procurement-defensible pick when Datadog is already the org-wide APM standard and adding a separate LLM observability vendor triggers a vendor review. Datadog LLM Observability ships LLM traces, prompt + completion logging, cost tracking, and quality monitoring inside the Datadog platform alongside infrastructure + APM + logs + RUM. Same auth, same dashboards, same procurement contract. AI-bolted-on architecturally (Datadog was built for general-purpose APM and added LLM observability in 2024) but for Datadog-native shops the procurement story dominates the technical tradeoff. Premium pricing reflects Datadog's enterprise positioning.

9. New Relic AI Monitoring New Relic · APM-native · best for teams already on New Relic · 2024 GA

LLM monitoring bundled into New Relic — the procurement-defensible pick when New Relic is already the org-wide APM standard. New Relic AI Monitoring ships LLM traces, prompt + completion capture, cost + latency tracking, and quality signals inside the New Relic platform alongside APM + infra + logs. Same usage-based pricing model as the rest of New Relic. AI-bolted-on architecturally (New Relic was built for general-purpose APM and added LLM monitoring in 2024) but for New Relic-native shops the procurement story dominates. Less mature LLM-specific feature set than Datadog's offering as of 2026, but improving.

10. Traceloop / OpenLLMetry Open-source (Apache 2.0) · OpenTelemetry-based · vendor-neutral · best for teams that want standards-compliance + multi-vendor portability

The OpenTelemetry-based vendor-neutral LLM observability standard — the right pick when 'I want to instrument once and route to any observability backend without vendor lock-in' is the bar. OpenLLMetry is an open-source Apache 2.0 OpenTelemetry extension that defines semantic conventions for LLM spans (LLM call, tool call, retrieval, RAG step). Instrument once with OpenLLMetry SDKs, route to Traceloop's hosted backend OR Datadog OR New Relic OR Honeycomb OR Langfuse OR any OTel-compatible backend. Traceloop is the company stewarding OpenLLMetry + offering a hosted backend. AI-baked-in for the spec; standards-first architecturally.

🚀 If you're a Solo founder shipping an AI feature into production this week

Your problem: You're a solo or 2-3 person team shipping an AI feature into production THIS WEEK. RAG over docs, semantic search, LLM-powered workflow. You need observability you can wire in 60 seconds — see traces, see costs, see if anything is broken — and won't have to migrate off in 6 months. Pair this decision with the AI Infrastructure megapage for the model substrate decision.

1. Helicone — 1-line proxy URL change = working observability + cost tracking in 60 seconds; fastest install in the category
2. Langfuse — free hosted tier generous + most complete OSS feature set; the substrate that grows with you from solo to enterprise
3. LangSmith — if you're shipping with LangChain, this is the zero-glue first-party tracing layer
4. Arize Phoenix — Apache 2.0 OSS that runs as a notebook companion locally — $0 cost, zero hosted dependency
5. Braintrust — if you're shipping a feature where regression matters from day one, start the eval discipline early

📈 If you're a Series A startup with 1-10K LLM calls/day needing eval discipline

Your problem: You have product-market fit and AI features in production. 1-10K LLM calls/day, real customer impact when an answer is wrong, prompt + model changes are happening weekly. You need real eval discipline — offline test suites, CI integration, A/B model comparison, golden datasets — not just traces. Pair with the Autonomous Coding Agents megapage for the build-velocity layer that ships prompt changes daily.

1. Braintrust — deepest evals layer in the category — offline + online + CI + A/B + golden datasets; built for this exact use case
2. Langfuse — evals + traces + prompts + cost tracking in one OSS-or-hosted tool; the most complete feature set without going evals-only
3. LangSmith — if you're on LangChain, the LangChain-native eval framework + first-party tracing is the procurement-defensible pick
4. Arize Phoenix — Apache 2.0 OSS with strong eval framework + multi-framework support — the OSS path if you want full control
5. Helicone — if cost tracking + caching are the load-bearing axis at this stage and evals can come later

🏢 If you're a Mid-market AI team with 100K-10M calls/day needing cost + quality monitoring

Your problem: You're 50-500 employees with 100K-10M LLM calls/day in production. Cost discipline matters (LLM bills are now meaningful budget lines), quality regressions matter (one bad prompt change = customer support escalation), and your AI substrate has to clear a 4-12 week vendor onboarding process. SOC 2 Type II + DPA + data-residency + audit logs all in scope. Coordinate with the Compliance Authority Graph for SOC 2 / ISO 27001 / HIPAA / GDPR posture.

1. Langfuse — complete feature set (traces + evals + prompts + cost) + OSS self-host option for data control + strong hosted compliance posture
2. Braintrust — if eval discipline is the load-bearing axis at this scale — depth wins over breadth
3. Arize Phoenix — Apache 2.0 OSS + OpenTelemetry-native + multi-framework — the inspectability + portability pick
4. LangSmith — if LangChain is the org-standard framework, the procurement-defensible pick + enterprise self-host tier emerging
5. WhyLabs — if you're in a regulated industry (finance · healthcare) and drift monitoring + audit trail are load-bearing

🏛 If you're a Enterprise CTO standardizing LLM observability org-wide (security review · multi-team · compliance)

Your problem: You're 1000+ employees standardizing LLM observability infrastructure org-wide. Multiple AI teams, multiple frameworks (some on LangChain, some on raw OpenAI/Anthropic SDKs, some on LlamaIndex), multi-cloud reality. Strict procurement, central FinOps, audit + compliance + DPA + BAA. You're picking the substrate the next 5 years of AI products will be monitored with — AI-baked-in vs AI-bolted-on matters at this horizon (see /operator cockpit for the operator-layer view).

1. Datadog LLM Observability — if Datadog is already org-wide APM standard, one-pane-of-glass + bundled procurement + cleared compliance posture wins
2. Langfuse Enterprise — best-in-class AI-native LLM observability with self-host + dedicated CSM tier — feature-velocity bet
3. Traceloop / OpenLLMetry — OpenTelemetry vendor-neutral instrumentation = no lock-in; route to any backend; standards-compliance bet
4. WhyLabs — regulated industries (finance · healthcare · government) — drift monitoring + audit-trail discipline + LangKit safety signals
5. New Relic AI Monitoring — if New Relic is org-wide APM, the procurement-bundle pick at lower cost than Datadog typically