The agent era is here.

Engineering teams are shipping AI-powered products, deploying multi-agent systems, and trying to figure out what observability even means for non-deterministic systems.

We know our customers want visibility into entire conversations (multi-trace sessions over time) that include handoffs, failures, and retries. We also hear a consistent need for, “What changed after I shipped this update to my agent? Did it introduce new failures/regressions?” and gain visibility into comparing runs across agent versions, detecting shifts in latency, failures, cost, or tool usage, and identifying new failure patterns post-deploy. So, here are use cases Honeycomb supports today to help answer these questions. We also have a major launch coming in May for agent observability that will make it much easier to understand what your agents are up to.

Observability for AI systems: LLM and agent observability

You're paying 10x more for an LLM that's getting worse results. Honeycomb shows you this in seconds.

LLM costs are invisible… until they aren't.

When a product team embeds AI into a customer-facing feature, the default approach is to pick a model, ship it, and hope. Honeycomb gives you something better: a live, correlated view of token spend, user outcomes, and model performance across every model you're running, broken down by any dimension you care about.

In this demo, a storefront uses large language models to answer customer product questions. Honeycomb surfaces a damning finding: the lightweight Claude Haiku model is costing ten times more per interaction than GPT-4o mini. Not just that. User sentiment from Haiku responses is 13% worse, time to first token is slower, and cart conversion is lower. It's right there in your observability data, correlated with the actual user behavior that your business cares about.

This is what makes Honeycomb's approach to LLM observability different: you can analyze LLM/AI agent data in context alongside the other moving pieces in your system. Token counts, latency, model IDs, and thumbs-up/thumbs-down ratings are just spans and fields. You query them, group them, correlate them, and drill into individual traces the same way you do everything else in Honeycomb.

Catch hallucinations, bias, and drift in your LLMs before they reach your users

Objective metrics like token count and latency only tell part of the story. The harder question—whether this model is actually producing good, accurate, unbiased output—requires a different layer of evaluation. Honeycomb can integrate with AI evaluation frameworks like Braintrust so that subjective quality signals show up directly in your traces, alongside your cost and performance data.

In this demo, Braintrust's auto-eval library is wired into the application, adding evaluation spans directly to traces. Those spans carry hallucination, bias, and relevance scores for each model interaction. In Honeycomb, you can aggregate across all your evals at a high level (how is Claude Sonnet 4.6 doing on hallucination this week compared to last?) and drill into any single conversation to see what happened, in full context.

The result: when Claude Sonnet 4.6 starts hallucinating and showing bias in a particular scenario, you don't find out from a user complaint. You find out from a graph, and you're already in the trace by the time anyone notices.

Your ticket triage agent is taking 14 minutes per ticket. Here's exactly why.

Agentic systems are black boxes by default.

A tool call happens, then another, then another, and somewhere in that chain your agent slows to a crawl. Without observability, you're guessing. But with Honeycomb, you can see every tool call, LLM invocation, and evaluation span, and immediately spot the one costing you 800 seconds of wall-clock time.

In this demo, an engineer receives a report about a slow ticket triage agent. A trace for a specific ticket ID reveals the issue instantly: LLM evaluation spans are running sequentially inside the main agent loop, each one taking four times longer than the actual work it was evaluating. These diagnostic evals were designed to run separately, not synchronously inside the agent's hot path. The result: over 800 seconds of unnecessary latency across the trace.

The fix is to run the evals outside the agent loop, in parallel or after the fact. That's a one-line architectural change that only becomes obvious once you can see what's actually happening inside your agent at the span level. Distributed tracing applied to AI is more important than ever.

Your multi-agent orchestrator times out and retries. Honeycomb exposes the pattern instantly.

Multi-agent architectures introduce failure modes that traditional monitoring was never designed to handle.

When one agent calls another and that call silently times out at exactly five minutes every single time, the root cause isn't a bug in your code. It's a misconfigured timeout. Honeycomb lets you see this pattern across all concurrent agent interactions at once, with enough detail to be certain.

In this demo, an orchestrator agent is failing to get responses from a sub-agent called "observability expert." Each failure records an error at exactly 300 seconds. By looking at the same span across all traces in the same time window, the pattern becomes obvious: dozens of spans, all failing at the same duration, all with the same "fetch failed" error. Now we have the evidence to work on a fix: increase the timeout in the short term, and work on speeding up the sub-agent.

Next up: power agentic observability with Honeycomb

We’ve seen how Honeycomb provides unique visibility into LLMs operating in your production environment. In the next post, we’ll dive into how Honeycomb provides observability insights uniquely suited to helping your AI agents rapidly diagnose and fix production issues, and build production feedback into the next round of development.