Feature Focus: A Six-Week Progress Recap - September 2024
Over the past six weeks, we introduced a series of impactful updates aimed at making your observability workflows faster, more unified, and more collaborative. Here’s a...
A CoPE’s Duty: Indexing on Prod
Building a center of production excellence (CoPE) starts with indexing on production. Here’s why. Odds are that a software engineer today is really focused on...
The Evolution of Engineering and the Role of Observability 2.0 in Shaping the Future
Engineering has come a long way since the days of delivering discrete, point-in-time products that were often packaged on a CD and shipped to customers....
Always. Enable. Keepalives.
As part of our recent failure testing project, we ran into an interesting failure mode involving the OpenTelemetry SDK for Go. In this post, we’ll...
A CoPE’s Guide to Alert Management
Alerts are a perennial topic, and a CoPE will need to engage with them. The bounds of this problem space are formed by two types...
The CoPE and Other Teams, Part 2: Custom Instrumentation and Telemetry Pipelines
The previous post laid out the basic idea of instrumentation and how OpenTelemetry’s auto-instrumentation can get teams started. However, you can’t rely only on auto-instrumentation....
Apdex in Honeycomb
“How is my app performing?” is one of the most common, yet hardest questions to answer. There are myriad ways to measure this, like error...
Making Room for Some Lint
It’s one of my strongly held beliefs that errors are constructed, not discovered. However we frame an incident’s causes, contributing factors, and context ends up...
The CoPE and Other Teams, Part 1: Introduction & Auto-Instrumentation
The CoPE is made to affect, meaning change, how things work. The disruption it produces is a feature, not a bug. That disruption pushes things...
Destroy on Friday: The Big Day 🧨 A Chaos Engineering Experiment - Part 2
In my last blog post, I explained why we decided to destroy one third of our infrastructure in production just to see what would happen....
What Makes for a 'Good' Pair Programming Session?
Software changes so rapidly that developing on the cutting edge of it cannot fall to a single person. When it comes to asynchronously disseminating information...
Deploy on Friday? How About Destroy on Friday! A Chaos Engineering Experiment - Part 1
We recently took a daring step to test and improve the reliability of the Honeycomb service: we abruptly destroyed one third of the infrastructure in...
Staffing Up Your CoPE
Getting the right people working in the CoPE is crucial to success because these change agents must limber up the organization and promote the flexibility...
Navigating Software Engineering Complexity With Observability
In the not-too-distant past, building software was relatively straightforward. The simplicity of LAMP stacks, Rails, and other well-defined web frameworks provided a stable foundation. Issues...
Framework for an Observability Maturity Model: Using Observability to Advance Your Engineering & Product
Everyone's talking about “observability,” but many don’t know what it is, what it’s for, or what benefits it offers. With this framing of observability in...