Technical Deep Dives

October 7, 2026

OpenTelemetry Collector Configuration for LLM Observability

A complete, annotated OpenTelemetry Collector configuration for LLM observability: receiving traces over OTLP, normalizing OpenInference/OpenLLMetry schemas onto GenAI semantic conventions, redacting sensitive prompt and completion content, managing volume without sampling away conversations, exporting to Honeycomb, plus validation steps and a troubleshooting table.

Instrumentation

Technical Deep Dives

Abstract illustration of an OpenTelemetry Collector funneling colorful data to a Sampling Processor.

September 21, 2026

Mike Goldsmith

How Adaptive Tail Sampling Works in the OpenTelemetry Collector

A technical deep dive into Honeycomb's adaptive tail sampling processor for the OpenTelemetry Collector: how decisions get made, the samplers available, how thresholds compose with the rest of a sampling pipeline, performance benchmarks, deployment limitations, and how it compares to Refinery.

Sampling

Technical Deep Dives

Graviton5 in Production at Honeycomb: Per-service Results From the m8g to m9g Migration

June 10, 2026

Liz Fong-Jones

Graviton5 in Production at Honeycomb: Per-service Results From the m8g to m9g Migration

This is the fourth installment in the Graviton retrospective series we've been writing since 2021. The methodology is the same one I always reach for: hold the workload constant, run both generations on the same Kubernetes namespace concurrently, and let the per-pod numbers speak.

Dogfooding

Technical Deep Dives

Making Semantic Conventions Work for You With OpenTelemetry Weaver

May 11, 2026

Mike Goldsmith

Making Semantic Conventions Work for You With OpenTelemetry Weaver

In this post, I'll show how OpenTelemetry Weaver lets you define, layer, and validate semantic convention schemas so that both humans and agents have full context on every attribute in your telemetry.

May 4, 2026

Taming Log Noise With the OpenTelemetry Collector's Drain Processor

Do you receive 50 million log lines per day and struggle to see what actually matters? Health checks, heartbeat pings, connection pool messages—they all drown out the errors and anomalies you're trying to find. What if the Collector could figure out the patterns for you?

April 23, 2026

Managing OpenTelemetry Semantic Convention Migrations With the Collector

This is what a semantic convention migration looks like in practice: not a clean cutover, but months of coexistence where old and new attribute names overlap. In this post, I'll explain why this happens, how the OpenTelemetry Collector's schema processor is designed to automate migrations in both directions, and what we're actively working on to get it into a state where everyone can use it.

January 6, 2026

How Time Series Databases Work—and Where They Don’t

In my previous post, we explored why Honeycomb is implemented as a distributed column store. Just as interesting to consider, though, is why Honeycomb is not implemented in other ways. So in this post, we’re going to dive into the topic of time series databases (TSDBs) and why Honeycomb couldn’t be limited to a TSDB implementation.

January 14, 2025

The Most Important Developer Productivity Metric

At Honeycomb, we believe in ensuring an awesome developer experience for our own engineering teams; it’s the only way that we can compete with companies that have way more engineers than we do. We’ve historically tried to keep our build times under 15 minutes, taking percussive action when build times got over 15 minutes.

Software Engineering

Technical Deep Dives

Observable Frontends: the State of OpenTelemetry in the Browser

April 27, 2023

Jessica Kerr (Jessitron)

Observable Frontends: the State of OpenTelemetry in the Browser

The modern standard for observability in backend systems is: distributed traces with OpenTelemetry, plus dynamic aggregations over these events. This works very well in the world of web servers. But what about the web client?

Observability

Technical Deep Dives

Achieving Great Dynamic Sampling with Refinery

April 18, 2023

Irving Popovetsky

Achieving Great Dynamic Sampling with Refinery

Refinery, Honeycomb’s tail-based dynamic sampling proxy, often makes sampling feel like magic. This applies especially to dynamic sampling, because it ensures that interesting and unique traffic is kept, while tossing out nearly-identical “boring” traffic. But like any sufficiently advanced technology, it can feel a bit counterintuitive to wield correctly, at first.

Sampling

Technical Deep Dives

November 18, 2022

Nathan Lincoln

Scaling Ingest With Ingest Telemetry

With the introduction of Environments & Services, we’ve seen a dramatic increase in the creation of new datasets. These new datasets are smaller than ones created with Honeycomb Classic, where customers would typically place all of their services under a single, large dataset. This change has presented some interesting scaling challenges, which I’ll detail in this post, along with the solution we used, and how we leveraged Honeycomb’s own telemetry to scale Honeycomb.

Dogfooding

Technical Deep Dives

January 11, 2022

Mike Goldsmith

What Is Auto-Instrumentation?

Instrumenting your code generates telemetry that shows you how your system is performing, and whether your system is healthy.

Instrumentation

Technical Deep Dives

November 30, 2021

Liz Fong-Jones

Scaling Kafka at Honeycomb

When you send telemetry into Honeycomb, our infrastructure needs to buffer your data before processing it in our “retriever” columnar storage database. For the entirety of Honeycomb’s existence, we have used Apache Kafka to…

Technical Deep Dives