Virtualizing Our Storage Engine
Our storage engine, affectionately known as Retriever, has served us faithfully since the earliest days of Honeycomb. It’s a tool that writes data to disk...
How to Use Relational Fields: Some Nifty Use Cases
We recently introduced relational fields, a new feature that allows you to query spans based on their relationship to each other within a trace. This...
Introducing Relational Fields
Expanded fields allow you to more easily find interesting traces and learn about the spans within them, saving time for debugging and enabling more curiosity...
Ask Miss O11y: Observability vs BI Tools & Data Warehouses
You probably have already answered this before, but do you have a good rule of thumb for where o11y [observability] ends and BI [business intelligence]/data...
Ask Miss O11y: How Can I Add o11y to Databases?
How do we bring observability to the DB world? In the SQL Server world, you can marry up perfmon and extended event traces but is...
How Time Series Databases Work—and Where They Don't
In my previous post, we explored why Honeycomb is implemented as a distributed column store. Just as interesting to consider, though, is why Honeycomb is...
Understanding Lambda Sleep Cycles With CONCURRENCY
CONCURRENCY is now enabled for all customers. See our docs page for information about how it works and how to use it effectively. Questions or...
Why Observability Requires a Distributed Column Store
Honeycomb is known for its incredibly fast performance: you can sift through billions of rows, comparing high-cardinality data across thousands of fields, and get fast...
Data Availability Isn’t Observability
But it’s better than nothing... Most of the industry is racing to adopt better observability practices, and they’re discovering lots of power in being able...
Event Latency: What It Is and Why You Should Care
Recently, we added a new derived column function to Honeycomb, INGEST_TIMESTAMP(), which can help customers debug event latency and/or inaccurate timestamps. A meaningful minority of...
From "Secondary Storage" To Just "Storage": A Tale of Lambdas, LZ4, and Garbage Collection
When we introduced Secondary Storage two years ago, it was a deliberate compromise between economy and performance. Compared to Honeycomb’s primary NVMe storage attached to...
Stop Your Database From Hating You With This One Weird Trick
Let's not bury the lede here: we use Observability-Driven Development at Honeycomb to identify and prevent DB load issues. Like every online service, we experience...
Speeding Things Up So Your Queries Can Bee Faster
Honeycomb strives to be a fast, efficient tool; our storage back-end satisfies the median customer query in 250ms (and the P90 in 1.3 seconds). Still,...
Postmortem: RDS Clogs & Cache-Refresh Crash Loops
On Thursday, October 4, we experienced a partial API outage from 21:02-21:56 UTC (14:02-14:56 PDT). Despite some remediation work, we saw a similar (though less...
There And Back Again: A Honeycomb Tracing Story
In our previous post about Honeycomb Tracing, we used tracing to better understand Honeycomb's own query path. When doing this kind of investigation, you typically have...