The most common state of a production AI agent today is opaque. It returns answers, customers (or internal users) interact with it, and the team behind it has almost no visibility into why any specific answer came out the way it did. When something goes wrong, a hallucinated number, an off-policy claim, a latency spike, the post-mortem is a guessing game.
This isn't a tooling gap; it's a discipline gap. The same teams that wouldn't dream of running a payments service without traces, metrics, and logs ship LLM-powered agents with none of the equivalents. AI agent observability is the layer that fixes that, and it's not the same as your existing APM stack, no matter what vendor marketing claims.
Why Generic APM Tools Aren't Enough
Datadog, New Relic, and the rest of the APM category are great at telling you that a request was slow or that a database query returned an error. They are not designed to answer questions like:
- What prompt was sent to the model for this specific user interaction?
- Which retrieved documents were included in the context window?
- What did the model output token-by-token before the post-processing layer rewrote it?
- Did the post-LLM judge flag this response, and if so, why?
- How does this interaction compare to similar interactions over the last 30 days?
You can hack some of this into a generic APM with custom spans and tags, but you end up with logs that work for engineers and nothing that works for the people actually running the agent, product owners, content writers, compliance reviewers. An agent observability layer is purpose-built for that audience, with the LLM call as the first-class object.
What to Instrument, The Five-Layer View
A production-grade observability stack for an AI agent captures five layers, end to end, for every interaction:
1. The Trace
Every user interaction generates a trace: the inbound request, the classifier verdict, the retrieval calls, the prompt assembly, the model call (or chain of model calls), the judge verdict, and the final response. The trace is the audit trail. Without it, you cannot reconstruct what happened, let alone fix it.
2. Inputs and Outputs
Capture the full input and output of each step. Yes, this means storing prompts and responses, not just metadata. Without the actual text, debugging is impossible. PII redaction happens at write time so the captured payloads are safe to store.
3. Cost and Latency Per Step
Token counts and dollar cost on every model call. P50, P90, P99 latency by step. This is how you catch silent regressions, a retrieval that's pulling 30% more chunks than it used to, a prompt that's grown 40% in tokens because someone added examples and forgot to trim, a model that's quietly slower this week than last week.
4. Quality Signals
The post-LLM judge's structured verdicts, written to the trace alongside everything else. Faithfulness scores, off-policy flags, PII detections, suggested actions. Aggregated over time, this is your single most valuable dataset, it tells you what's actually breaking, before users complain.
5. User Feedback
Thumbs up and thumbs down attached to the original trace. The teams who do this well also surface a comment box on negative feedback, and route the comment plus the trace to whoever owns that intent. The feedback loop closes when those comments drive prompt and retrieval improvements that show up in the quality signals.
The Self-Hosted Question
The big open-source player in this space is LangFuse, and it's a reasonable starting point for many teams. The decision worth deliberating is whether to run it self-hosted or on the vendor's cloud.
Self-hosted is the right choice when:
- You're storing prompts and responses that contain regulated data (healthcare, financial, legal).
- Your security team requires data to stay in your VPC or specific region.
- You expect heavy volume and the cloud pricing math doesn't work.
The standard self-hosted pattern is the LangFuse container plus a managed Postgres plus S3 for blob storage of larger payloads (long prompts, full transcripts, retrieved document chunks). This setup is two days of work for an experienced infra engineer, including network policies and access controls.
The vendor-hosted version is fine for non-regulated, lower-volume workloads. The operational overhead is essentially zero. The cost crossover for most teams happens somewhere in the high thousands of dollars per month, before that, self-hosting is a tax, not a saving.
What the Dashboards Should Show, and Who Should Use Them
An observability layer that only engineers use is half-built. The audiences worth instrumenting for:
Engineers and ML Practitioners
They want to see individual traces. Click any user interaction, walk through every step, see exactly what the model received and returned. This is the debugging surface.
Product Owners
They want aggregated views. Volume by intent, satisfaction by intent, latency P99 trends, cost per resolved conversation. The questions are commercial, is this thing earning its keep, not technical.
Content and Prompt Engineers
They want side-by-side comparisons. The same input, this week vs last week, before and after a prompt change. Token diffs on the system prompt. They also want a workbench where they can replay a failed trace with a tweaked prompt and see if the new version would have produced a better answer.
Compliance Reviewers
They want sampled audits. A weekly sample of interactions, filtered by the off-policy flags from the judge, with the ability to override the verdict and add notes that feed back into the eval suite. This is the slowest-moving but most senior audience, and the value of having them in the system is proportional to how regulated your industry is.
Common Failure Modes We've Seen
Logging the Wrong Thing
Teams turn on verbose logging and end up with terabytes of model invocations they can't search. The fix is to capture traces (one per interaction) with structured spans, not raw log lines. The query interface is built for traces.
No Sampling Strategy
If every interaction goes through every quality check, costs scale linearly with usage. A reasonable default: 100% of traces are stored, the cheap post-LLM judge runs on 100%, the expensive judge runs on a sampled 5%, and human review pulls from the judge-flagged subset. This gives you full audit coverage and bounded cost.
Skipping the Feedback Loop
The point of observability is not to collect data; it's to improve the system. If you don't have a cadence (weekly, bi-weekly) where the team reviews flagged traces and converts them into prompt updates, retrieval improvements, or new eval cases, you're paying for storage and getting nothing back.
Treating the Eval Suite as a One-Time Build
The evaluation set that ships with v1 of an agent is stale within 90 days, user behavior shifts, the product expands, new failure modes appear. Every flagged trace that turned out to be a real issue should become a permanent eval case. The set grows with the system, and the team's confidence in shipping changes grows with it.
How to Start in a Day
You don't need a six-week project to get the first version of observability shipping. The minimum viable layer:
- Stand up a trace store. LangFuse self-hosted, two containers, or the vendor cloud if your data is non-regulated.
- Wrap your model calls. Every LLM invocation logs its input, output, model, latency, and token cost as a span on a trace.
- Tag traces with intent and user. So you can filter and aggregate.
- Add thumbs up/down on the UI. Wire the feedback into the trace.
- Set up a weekly review. One hour, the team looks at thumbs-down traces and the highest-cost interactions, and decides what to fix.
The judge integration, the prompt diff tooling, the compliance sampler, those come later. The first five steps are what unlocks every later improvement.
Frequently Asked Questions
Is AI agent observability the same as LLM observability?
They overlap heavily but agent observability is broader. LLM observability focuses on model calls. Agent observability adds the orchestration around them, tool use, retrieval, classifiers, judges, and the resulting user-visible decision, into one trace.
Why not just use OpenTelemetry for everything?
You can, and the underlying storage often is OpenTelemetry-compatible. But the prompt-aware tooling on top, side-by-side prompt comparisons, eval replay, judge integration, is what makes the layer useful for non-engineers. OpenTelemetry alone gives engineers what they need and gives nobody else anything.
How much does this cost to run?
Self-hosted on a small Postgres plus S3 typically runs under a few hundred dollars per month for low-to-mid volume agents. Vendor-hosted plans scale with traces; budget a low four-figure monthly spend for production-scale B2B agents.
Do we need observability if we're using a no-code agent builder?
You need it more, not less. No-code builders hide the prompts, the retrieval choices, and the post-processing from you. Without observability you have no way to debug the parts of the system you didn't write.
How does observability relate to guardrails?
Guardrails are the runtime defenses (classifier, judge) that prevent bad outcomes. Observability is the persistent record of every decision those guardrails made, plus everything they couldn't catch. Guardrails block; observability learns. You need both.
If you're running an AI agent in production and can't answer the question "why did it say that," you're operating blind. See how we instrument production AI systems or talk to us about your current stack.