Login 
LoginIncident Report: Running Dry on Memory Without Noticing
On November 6, 2019, we intermittently rejected 1-3% of customer telemetry data at ingest for four periods of 20 minutes each. The trigger of the...
OpenTelemetry vs OpenTracing | Understanding OpenCensus and Related Terms
There’s been a fair bit of buzz lately about OpenTelemetry, which is the next major version of the OpenTracing and OpenCensus projects. The leadership of...
Working Toward Service Level Objectives (SLOs), Part 1
In theory, Honeycomb is always up. Our servers run without hiccups, our user interface loads rapidly and is highly responsive, and our query engine is...
Working On Hitting a Release Cadence? CI/CD Observability Can Help You Get There
We recently sponsored our partner CloudBees' conference DevOps World & JenkinsWorld in San Francisco and our message “Observe how Customers Experience Your Build” resonated well...
The Path from Unstructured Logs to Observability
Are you starting out on your journey toward observability? Do you have a mandate from management, or are you a lone warrior in the matrix?...
Never Alone On Call
Does your organization have an on-call rotation? Several members of the Honeycomb engineering team recently hosted a live webcast about why they never feel alone...
Taming A Game-Changer: Honeycomb and GraphQL at VendHQ
This guest post is from Evan Shaw, Lead Engineer at vendhq.com. GraphQL: a game-changer GraphQL is a query language for APIs. It allows you to...
Velocity (& Reliability) - Two must-haves for every software engineering team
(Field notes from O’Reilly’s Velocity 2019 Show, San Jose.) It was steamy hot in San Jose during O’Reilly’s Velocity show and the normally frigid AC...
Making Instrumentation Extensible
Observability-driven development requires both rich query capabilities and sufficient instrumentation in order to capture the nuances of developers' intention and useful dimensions of cardinality. When...
Toward a Maturity Model for Observability
Access to observability is becoming critical to organizations shipping software, running modern infrastructures in production, and to understanding how users are experiencing their service. To...
Dynamic Sampling by Example
Last week, Rachel published a guide describing the advantages of dynamic sampling. In it, we discussed varying sample rates to achieve a target collection rate...
Anatomy of a Cascading Failure
In Caches Are Good, Except When They Are Bad, we identified four separate problems that combined together to cause a cascading failure in our API...
When In Doubt, Add More Spans: A Tale of Tracing and Testing In Production
Recently, Toshok was telling a story about the kind of thing he talks about a lot—improving the performance of some endpoint or page or other....
Incident Review: Caches are Good, Except When They Are Bad
Between Wednesday, April 17th and Friday, April 26th, Honeycomb had four separate periods of downtime affecting the Honeycomb API, resulting in approximately 38 minutes of...
A New Bee's First Oncall
I'm Honeycomb's newest engineer, now on my eighth week at Honeycomb. Excitingly, I did my first week of oncall two weeks ago! Almost every engineer...