What is AI observability?

Why classic monitoring is not enough

Application performance monitoring was built for systems whose correctness is checkable: a request either returned the right status in the right time or it did not. AI systems break that assumption in three ways. The output is non-deterministic, so the same input can produce different answers. Correctness is a matter of degree and context, not a status code. And the expensive, failure-prone part of the system is a third-party model whose internals you cannot see.

The result is a new class of incident that traditional monitors miss entirely: hallucinated answers, prompts that leak personal data, agents that loop through tool calls until a budget is exhausted, and a model version change at the provider that quietly degrades quality. Each of these is visible only if the content, not just the metadata, is captured and judged.

The signals of AI observability

The three classic pillars still apply; AI adds specific fields to each and one new pillar of its own.

• Traces. One span per model call and tool call, nested under the user request, carrying the prompt, response, model, token counts and latency. The OpenTelemetry GenAI semantic conventions define the attribute names so tools can interoperate.
• Metrics. Request rate, error rate, latency percentiles, time to first token, tokens per request and cost per request, broken down by model, provider, team and feature.
• Logs. The full record of each request where policy allows storing prompts and responses, with redaction where it does not, so an incident can be replayed.
• Quality signals. Scores from automated evaluators, human review and user feedback attached to traces, which is the pillar classic observability lacks.
• Cost and usage. Spend attributed to the identity that caused it, which is what makes budgets and chargeback possible.

Where to capture telemetry

There are three places to capture AI telemetry, and mature setups use two of them. In the application, through an SDK or framework callbacks, which sees everything the code sees, including retrieval steps and intermediate reasoning, but has to be added to every application and kept consistent. At a gateway, which sees every model and tool call that passes through it with no code change, with the caller identity already resolved, but does not see steps that happen inside the application. Through a proxy or sidecar, which is the gateway pattern applied per service.

The practical split is that the AI gateway provides complete, consistent coverage of what left the building and what it cost, and application instrumentation adds the inside view for the systems where quality matters most. Both should export in the same conventions so one trace shows both.

Observability, evaluation and monitoring are different jobs

Observability answers what happened in production: the trace, the cost, the latency, the tool calls. Evaluation answers whether the output was good, by running judges, rubrics or test suites over outputs, in production samples or in offline datasets before a release. Monitoring watches a few numbers continuously and pages someone when they move. The three share data and are often sold together, but they are owned by different people and answer different questions.

Where they meet is the feedback loop: production traces become evaluation datasets, evaluation scores become monitored metrics, and monitored regressions are investigated with traces. Tooling that supports that loop end to end is what most teams mean when they say an AI observability platform.

What changes for agents

An agent turns one request into a loop of model calls and tool calls whose length is decided at run time. Observability has to capture the loop as one trace, show which tool was called with which arguments and what came back, and surface the patterns that only appear across runs: loops that never terminate, tools that fail silently, costs that scale with the number of steps rather than the number of users.

Tool calls are also where the security questions live. Which agent read which record, as whom, is an audit question before it is a quality question, which is why gateways that broker tool calls log them with the caller identity. See What is an MCP gateway?.