How to Track Token Cost Across LLM Workflows

AI features rarely fail because of model quality alone. They often struggle when teams can’t predict the cost of running them in production. “Token cost” is what you pay an LLM provider for the input, output, and sometimes cached tokens processed during a request. Unlike crypto tokens, AI tokens are units of model usage, often chunks of text, that providers use to meter and price workloads.

For teams shipping AI products, token usage is a core unit-cost signal behind summarization, support answers, agent steps, code suggestions, and retrieval workflows. Without request-level tracking, you may know the monthly bill, but not which feature, customer, model, prompt, or retry loop caused it.

This guide shows how to track token usage and cost so teams can attribute spend, catch inefficiencies, and make better AI investment decisions.

How to track token cost step by step

The process for tracking token cost is straightforward:

1. Capture token usage from every model response.
2. Attach business metadata, such as feature, environment, team, user, or workspace.
3. Compute request cost using the provider’s pricing table at log time.
4. Store the result as request-level telemetry.
5. Build dashboards for cost trends, outliers, and cost per outcome.
6. Alert when usage drifts from expected baselines.

The goal is to move beyond monthly invoices and create visibility into where costs originate. Use this table as a guide to the kinds of fields you should capture and where they exist across your solution:

What data do you need to measure token cost accurately?

To track token costs accurately, you will need to log costs with the provider, model, and pricing version. To give finance and engineering the same receipt, be sure to compute the cost when the request is logged, not weeks later in a reporting job. Pricing tables, model names, cache rules, and tool charges can change quickly and unpredictably. Here’s what to know about measuring tokens.

Input tokens, output tokens, and cached tokens

Most AI token cost calculations need at least three usage values:

• Input tokens represent the prompt, retrieved context, tool instructions, and conversation history sent to the model.
• Output tokens represent the model’s generated response.
• Cached tokens represent repeated prompt material that providers may bill differently from fresh input.

This distinction between the types of token matters because provider pricing is not always a single rate. Most providers charge separately for different types of token usage. For example, OpenAI’s pricing documentation says tokens are billed at the chosen model’s input and output rates, and its pricing page lists cached input pricing where applicable. Anthropic’s pricing documentation separates base input tokens, cache writes, cache hits and refreshes, and output tokens.

Cost calculation from provider pricing tables

The basic formula for token costs should be:

Request cost = (input tokens x input rate) + (output tokens x output rate)

If rates are listed per million tokens, divide each token count by 1,000,000 before multiplying.

For some use cases, you can greatly reduce costs by utilizing prompt caching. Input tokens from cache read are usually heavily discounted (~90%). In this case, you will need to separate out cached and uncached input. You will also need to consider that some providers charge a premium to write to the cache, with the premium increasing the longer the input is cached. So use caching only where prompts are likely to be reused often enough to justify the write premium.

What you can safely leave out of cost logs

Many teams assume they need to store prompts to measure token cost. However, for most attribution and observability use cases, you only need the following:

• Token counts
• Model ID
• Request ID
• Feature names
• Environment tags
• User or workspace identifiers
• Success or error status

This is an important privacy boundary. Teams can measure cost per request, cost per feature, and token cost by user without storing raw prompts or sensitive customer content in their cost telemetry.

For deeper prompt behavior analysis, pair cost fields with an LLM token counter guide, but keep cost logs focused on the metadata needed to explain spend.

How should you instrument token cost by request, feature, and environment?

Monthly invoices tell you how much you spent, and request-level telemetry tells you why. Here’s how to instrument your token costs.

Capture usage at the model-call boundary

To make token cost actionable, capture usage data at the same place your application calls the LLM provider: the wrapper, gateway, or service function responsible for the model request. This is where you will have the provider response, application context, and trace context.

After the model call returns, read the provider’s usage data from the response and normalize it into fields your team controls. Do not build dashboards around each provider’s raw field names. Instead, map them into a consistent schema. The next section shows how to do this the standard way, using the OpenTelemetry GenAI semantic conventions, so your telemetry stays vendor-neutral and lights up purpose-built tooling like the Honeycomb Agent Timeline.

Tagging requests with feature and environment metadata

Provider invoices rarely know what your product was trying to do. Your application does. Enrich the event before you leave the service, and you can use a schema like this:

app.feature
app.environment
app.team
app.workspace_id
app.user_tier
app.experiment

These tags make it possible to visualize LLM cost per user, workspace, feature, or environment.

A minimum useful telemetry event might look like this:

{
 "name": "chat gpt-4o",
 "request_id": "req_123",
 "trace_id": "trace_abc",
 "gen_ai.operation.name": "chat",
 "gen_ai.request.model": "gpt-4o",
 "gen_ai.response.model": "gpt-4o-2024-08-06",
 "gen_ai.usage.input_tokens": 4210,
 "gen_ai.usage.cache_read_input_tokens": 1800,
 "gen_ai.usage.output_tokens": 640,
 "gen_ai.response.finish_reasons": ["stop"],
 "llm.request_cost_usd": 0.0187,
 "llm.pricing_version": "2026-06-01",
 "app.feature": "contract_summary",
 "app.environment": "production",
 "app.team": "documents",
 "app.workspace_id": "workspace_456",
 "duration_ms": 2430,
 "status": "success"
}

Emit this event once per LLM request, ideally as part of the same trace that captures retrieval calls, tool calls, retries, and downstream work. This gives teams enough context to debug cost alongside latency, errors, model changes, and product behavior.

Storing request-level cost telemetry for later analysis

Invoice totals are useful for accounting, but they are too late and too coarse for engineering action. Storing token cost as OpenTelemetry-style, request-level telemetry rather than as a monthly spreadsheet lets you ask questions like:

• Which model change increased output tokens?
• Which agent step is retrying?
• Which customer segment has the highest cost per outcome?
• Which feature has the best margin?

An AI observability platform should let you answer those questions without pre-aggregating away the context.

Speak the same language: OpenTelemetry GenAI Semantic Conventions

Here’s the thing about a homegrown schema: it works great right up until the moment someone else has to read it. One team calls it input_tokens, another calls it prompt_tok, a third stuffs everything into a blob, and now your token-cost dashboards need a decoder ring. The fix is to stop inventing field names and adopt a standard that the whole ecosystem already agrees on: the OpenTelemetry GenAI semantic conventions.

The GenAI semantic conventions are a shared vocabulary for describing LLM and agent behavior in telemetry, all namespaced under gen_ai.*. When you emit these attributes, any OpenTelemetry-native backend understands your data out of the box, you sidestep vendor lock-in, and you get to reuse instrumentation that libraries and SDKs already produce. In Honeycomb specifically, gen_ai.* attributes are what power the Agent Timeline, so instrumenting to the spec is what turns a wall of spans into a readable, per-conversation story of what your agent did and what it cost.

Attach these attributes to your GenAI spans

Add the following OpenTelemetry GenAI attributes to the spans your agents and model calls emit. Honeycomb uses them to group spans into conversations, identify which agent did what, and surface token and cost data where you actually investigate. The first three are required for the Agent Timeline to work its magic; the rest give you the cost and behavior detail:

Note that the base conventions cover input and output tokens directly. For prompt caching, providers and instrumentation also emit cache-specific usage counts, such as gen_ai.usage.cache_read_input_tokens and gen_ai.usage.cache_creation_input_tokens. Capture those separately so your cost math can apply the discounted cache-read rate and the cache-write premium rather than billing everything at the full input rate.

Map your old fields to the spec

Already shipping a custom llm.* schema? You don’t have to throw it out overnight. Here’s the straight-across translation from the custom fields used earlier in this guide to their GenAI-convention equivalents:

A useful rule of thumb: use the standard gen_ai.* attributes for everything the provider reports (tokens, model, finish reason), and keep your own namespaced attributes (like llm.request_cost_usd and app.feature) for the things only your business knows. The spec deliberately doesn’t define dollar cost, so you compute that at log time from the token counts and your versioned pricing table, then store it right next to the standard usage fields.

Name your GenAI operation spans consistently

Span names carry meaning too. Follow these patterns so the Agent Timeline can recognize each operation type and display it correctly:

Keep prompts and completions in span events (and off the cost path)

Full prompts, chat history, and completion responses can be large and may contain PII or other sensitive data, exactly the stuff you do not want sitting in cost telemetry. The conventions handle this cleanly by putting message content in span events rather than plain span attributes, so your OpenTelemetry Collector can filter or drop it before it ever reaches your backend:

• gen_ai.input.messages holds the chat history or input prompts sent to the model.
• gen_ai.output.messages holds the messages the model returned.
• gen_ai.evaluation.result attaches evaluation results to the operation span so you can review quality alongside cost.

This maps directly onto the privacy boundary described earlier: your cost logs keep token counts, model IDs, and business tags, while the bulky, sensitive message content lives in events you can scrub at the Collector. You get cost attribution and prompt-level debugging without one contaminating the other.

Get multi-agent attribution right

In a multi-agent workflow, correct attribution is the whole game. Two rules keep the timeline honest:

• Give every agent a unique gen_ai.agent.name. The Agent Timeline groups spans by this name, so duplicates or omissions make agents impossible to tell apart. Omit it and the span shows up as "Unknown." Sub-agents get their own distinct names rather than inheriting the parent’s.
• Let the caller emit the invoke_agent span, not the agent being called. The called agent then emits its own chat, execute_tool, and other spans under its own name, so each agent’s work (and its token spend) stays separated.

Record errors so failures don’t hide

Agentic workflows fail in sneaky ways: a tool call that silently drops its result, or an error that surfaces in a child span but never bubbles up to the parent. Record errors following the OpenTelemetry specification and include whatever applies: error.type / exception.type, error.message / exception.message, and error.stacktrace / exception.stacktrace. For tool-call failures, propagate the error status up to the parent span so a failed step never looks like a successful (but expensive) one.

Already emitting telemetry? Remap it at the Collector

If a tool emits OpenTelemetry-protocol telemetry but doesn’t yet use the GenAI semantic conventions, you don’t have to touch the source. The Collector’s transform processor can remap span names and attributes on the way through. For example, Claude Code emits telemetry over OTLP but with its own attribute keys; the following config remaps it to the GenAI conventions so it lights up the Agent Timeline, including aliasing the bare token fields up to the standard gen_ai.usage.* names:

processors:
  transform:
    error_mode: ignore
    trace_statements:
      - context: resource
        statements:
          - set(attributes["service.name"], "<AGENT NAME>")
      - context: span
        statements:
          # Identity attributes on every span
          - set(attributes["gen_ai.agent.name"], "<AGENT NAME>")
          - set(attributes["gen_ai.conversation.id"], attributes["session.id"])
          # claude_code.interaction -> invoke_agent
          - set(name, "invoke_agent <AGENT NAME>")
              where name == "claude_code.interaction"
          - set(attributes["gen_ai.operation.name"], "invoke_agent")
              where name == "invoke_agent <AGENT NAME>"
          # claude_code.llm_request -> chat {model}
          - set(name, Concat(["chat ", attributes["gen_ai.request.model"]], ""))
              where name == "claude_code.llm_request"
          - set(attributes["gen_ai.operation.name"], "chat")
              where IsMatch(name, "^chat ")
          # claude_code.tool -> execute_tool {tool_name}
          - set(name, Concat(["execute_tool ", attributes["tool_name"]], ""))
              where name == "claude_code.tool"
          - set(attributes["gen_ai.operation.name"], "execute_tool")
              where IsMatch(name, "^execute_tool ")
          - set(attributes["gen_ai.tool.name"], attributes["tool_name"])
              where IsMatch(name, "^execute_tool ")
          # Alias bare token fields to the gen_ai.usage namespace
          - set(attributes["gen_ai.usage.input_tokens"],
              attributes["input_tokens"])
          - set(attributes["gen_ai.usage.output_tokens"],
              attributes["output_tokens"])
          - set(attributes["gen_ai.usage.cache_read_input_tokens"],
              attributes["cache_read_tokens"])
          - set(attributes["gen_ai.usage.cache_creation_input_tokens"],
              attributes["cache_creation_tokens"])

Statements run in order, so rename the span first, then set gen_ai.operation.name off the new name. Once the remap is in place, that previously opaque telemetry flows into the same conversation view as everything else, tokens included.

Bottom line: instrument to the OpenTelemetry GenAI semantic conventions, compute cost from the standard token fields at log time, and keep your business metadata in your own namespace. You get portable, tool-agnostic telemetry and a token-cost story that reads cleanly in the Agent Timeline instead of a pile of bespoke fields nobody remembers naming.

How do you turn token cost into a useful operating dashboard?

Signals like “monthly provider spend went up" are not as helpful to understanding feature ROI as signals that are more closely tied to outcomes. An example of a more useful signal would be “cost per successful support answer increased 38% after the new retrieval prompt shipped.” Here’s how to create a useful operating dashboard.

Cost trends and outliers

Start with five dashboard views:

• Spend over time
• Cost by feature
• Cost by model
• Cost by environment
• Highest-cost/outlier requests

A dashboard panel of outlier requests can often be the most useful view. A single runaway request, huge context window, or retry loop can distort daily spend. Seeing the most expensive traces helps engineers find the actual cause, not just the symptom.

Cost per feature and cost per outcome

One of the least useful metrics in AI operations is total monthly spend. A much better metric is cost per successful outcome.

Examples include:

• Cost per generated summary
• Cost per document processed
• Cost per completed workflow
• Cost per active user session

These metrics connect AI spending directly to business value for data-driven decision-making. For launches, experiments, and model migrations, the same dashboards help teams track API token usage and costs and compare cost per outcome before and after each change.

How can you keep token cost from scaling faster than feature value?

Instead of optimizing for the cheapest model or the fewest tokens, the team’s objective should be to optimize for the lowest cost per successful outcome. There are several optimization techniques that consistently deliver the biggest savings.

Cache repeated prompts and trim context windows

The highest-impact savings usually come from reducing repeated input. Cache stable prompt prefixes, system instructions, examples, policy text, and reusable documents when your provider supports it. Then trim the context window to include only what the model needs to complete the task.

Context window creep is one of the easiest ways for LLM token usage to grow. Retrieval results get longer, conversation history expands, tool outputs accumulate, and suddenly every request carries thousands of unnecessary input tokens. Because you captured gen_ai.usage.input_tokens and gen_ai.usage.cache_read_input_tokens separately, you can actually see this creep happening rather than guessing.

Route to the right model and cap output length

Not every request requires the most capable model. Simple classification, extraction, and routing tasks can often use smaller, less expensive models when quality is sufficient. Reserve larger or more expensive models for complex reasoning, high-value customers, or escalation paths.

Set output token caps, too. A single chatty response should not be able to blow up daily spend. Max output limits, stop conditions, and response-shape constraints keep costs predictable. Watch gen_ai.response.finish_reasons for ["length"] to catch responses that hit the ceiling.

Alert on retry loops, runaway agents, and context creep

The biggest cost spikes often come from systems doing the same thing repeatedly: retry loops, agents calling tools in circles, workflows that reprocess the same document, or sessions that keep appending context.

Set anomaly alerts on tokens per request, tokens per session, cost per feature, and cost per workspace. This is where observability cost management becomes operational: teams see drift while they can still fix it.

A production-ready checklist for operationalizing token cost tracking

Use this checklist to establish a production-ready token cost tracking program:

• Assign ownership to verify and capture provider pricing (this can be versioned and maintained in code).
• Require model, feature, environment, team, and user or workspace tags on every request.
• Log input tokens, cached input tokens, output tokens, request cost, latency, and finish reason.
• Create daily budget alerts and anomaly detection for tokens per session and cost per feature.
• Reconcile telemetry totals against provider invoices monthly.

How Honeycomb helps teams track token cost

Tracking token cost only matters if teams can understand the behavior behind the numbers. Honeycomb gives engineering teams request-level visibility across prompts, models, users, retrieval systems, tool calls, and agent workflows so they can investigate individual requests instead of relying on aggregate billing reports.

Because Honeycomb reads the OpenTelemetry GenAI semantic conventions natively, the gen_ai.* attributes you emit flow straight into the Agent Timeline, where token usage, cost, and failures line up as a single coherent conversation. That context helps teams identify costly patterns, understand which features are efficient, and connect AI spend to business outcomes. The result is greater predictability, stronger accountability, and faster investigation when costs unexpectedly increase.

Learn more about Honeycomb for LLM observability cost management.