Short answer: AI agent observability should capture the full path from user request to model call, tool call, retrieval step, policy decision, cost event, error, and final answer. If you only log the final model response, you cannot reliably debug failures, control spend, or prove that safety gates worked.
AI agents fail differently from traditional web applications. A normal service might return a 500 error, time out, or write a bad database row. An agent can do all of that, but it can also choose the wrong tool, retrieve the wrong context, ignore a policy, exceed a cost budget, cite the wrong source, or produce a plausible answer for the wrong reason.
That is why observability for AI agents needs more than generic application logs. You need traces, metrics, logs, and alerts that are designed around agent behavior.
What AI agent observability should answer
A production observability system should let an operator answer these questions quickly:
- What did the user ask?
- Which model, prompt version, and tool definitions were used?
- Which tools or retrieval systems did the agent call?
- What inputs and outputs moved through each step?
- Where did latency and token cost accumulate?
- Did the agent hit policy, authorization, or human-approval gates?
- Was the final answer grounded in retrieved evidence?
- Which version of the system produced a bad or expensive result?
If the answer to any of these questions is “we would need to reproduce it manually,” the observability design is incomplete.
1. Trace the full agent run
Start with traces. A trace should represent one user-facing agent run from request to final output. Inside that trace, capture spans for the major steps: routing, retrieval, model calls, tool calls, guardrail checks, approval waits, retries, and final response generation.
This structure matters because agent failures are usually multi-step. A bad answer might come from poor retrieval, a stale prompt, a tool timeout, a bad retry, or a model hallucination after the right evidence was retrieved. Without trace-level structure, those causes collapse into one vague “the model failed” event.
Useful trace attributes include:
- Agent name and version.
- Prompt template version.
- Model provider and model name.
- Tool name, arguments, result status, and duration.
- Retrieved document IDs or chunk IDs.
- Policy gate result and approval state.
- Token counts, estimated cost, and latency per step.
The OpenTelemetry GenAI semantic conventions are a useful reference point because they define common telemetry concepts for generative AI operations, including model calls, token usage, and operation-level attributes. Even if you use a vendor tool, aligning internal names with common conventions reduces future migration cost.
2. Capture metrics that map to real risk
Dashboards should not only show request volume. For agents, the most useful metrics are tied to product reliability, cost, safety, and user impact.
Track at least:
- Task success rate: percentage of runs that completed the intended workflow.
- Tool error rate: failed tool calls, timeouts, malformed arguments, and denied authorizations.
- Retrieval quality signals: empty retrievals, low-confidence retrievals, and citations missing from answers.
- Latency: end-to-end latency plus per-step latency for model calls, retrieval, tools, and approvals.
- Token and cost usage: input tokens, output tokens, cached tokens where relevant, and estimated spend per run.
- Retry rate: repeated model calls, repeated tool calls, and fallback model usage.
- Safety intervention rate: blocked outputs, blocked tool calls, escalation to humans, and policy refusals.
Cost metrics are not optional. A small percentage of agent runs can consume a disproportionate amount of spend when loops, retries, long contexts, or expensive models are involved.
3. Log inputs and outputs carefully
Logs should capture enough detail to debug behavior, but not so much that they create a privacy or security problem. This is a design tradeoff, not a checkbox.
At minimum, log:
- Request ID and user/session identifier using privacy-safe IDs.
- Agent version, prompt version, and model version.
- Tool names, normalized arguments, result status, and error messages.
- Retrieval source IDs and metadata, rather than full sensitive documents where possible.
- Policy decisions, approval decisions, and refusal reasons.
- Final answer metadata, including whether citations were present.
For sensitive domains, avoid storing raw prompts or raw tool outputs by default. Use redaction, hashing, access controls, retention limits, and environment-specific sampling. Observability should not become a new data leak.
4. Alert on behavior, not just infrastructure
Traditional alerts catch CPU saturation, queue backlog, and HTTP errors. AI agents also need behavior-level alerts.
Examples:
- Cost per run exceeds a threshold.
- Tool-call error rate spikes after a deployment.
- More than a fixed percentage of runs hit retry loops.
- Safety blocks or human escalations suddenly drop to zero.
- Retrieval returns no useful context for common questions.
- Latency increases in a specific model, retriever, or tool span.
- Citation coverage falls below an expected baseline.
The “drop to zero” case is important. A sudden disappearance of safety blocks or approval events can indicate that the gate is no longer running, not that the system became safer overnight.
5. Connect observability to evaluation
Observability and evaluation should reinforce each other. Evaluation tells you whether a system should be released. Observability tells you whether the released system is behaving as expected.
Production traces should feed future test cases. When a user reports a poor answer, convert that trace into a regression example. When a cost spike happens, add a budget test. When a retrieval miss happens, add it to the golden dataset.
For related release-gate guidance, see our AI agent evaluation framework and our RAG evaluation checklist.
6. Keep a minimum production checklist
Before putting an AI agent into production, verify the following:
- Every user-facing run has a trace ID.
- Each model call records model name, prompt version, token usage, latency, and status.
- Each tool call records tool name, normalized arguments, result status, duration, and authorization result.
- Retrieval steps record source IDs, chunk IDs, rank, and retrieval status.
- Safety gates record allow, block, escalate, and human-approval outcomes.
- Cost is attributable by run, user segment, workflow, model, and tool path.
- Dashboards show task success, tool failures, latency, retry rate, safety interventions, and cost.
- Alerts exist for cost spikes, error spikes, latency spikes, retry loops, and missing safety events.
- Logs have privacy controls, access controls, and retention limits.
- Bad production traces can be converted into regression tests.
Common implementation mistakes
- Logging only the final answer. This hides the steps that caused the answer.
- Not versioning prompts. Without prompt versions, regressions are hard to attribute.
- Mixing all failures into one error metric. Retrieval misses, tool errors, policy blocks, and model refusals require different fixes.
- Ignoring cost until the bill arrives. Token and retry metrics should be visible during normal operation.
- Storing too much sensitive data. Raw logs can create new compliance and security exposure.