Debugging   Events   Observability  

Event Foo: A Series of Unfortunate/Incredible Events

By Charity Majors  |   Last modified on August 23, 2021

Good technical intuition is one of the things that defines a good senior engineer. And unpacking that intuition is the most valuable teaching tool. By making your implicit assumptions and experiences explicit, others can learn from them.

Lately I’ve been having a lot of conversations with people about debugging with events, or structured log data, with a dawning realization that this is not a universal experience. What I thought was as normal as code reviews is an intuition fed by some highly specific experiences.

Events seem ordinary and intuitive to many engineers—that’s how our brain works, after all—but not all of us. Furthermore, many engineers in the ops space have spent years or decades learning, intentionally and painstakingly, to retrain their brains to think in terms of metrics and aggregates. Which does not come as naturally, but is no less sticky once learned.

What makes an event useful or good? What makes it not-useful, or even deceptive? Instead of assuming full knowledge, let’s assume none and start from scratch.

We have invited some of our favorite people to start at the beginning and unpack their experience and understanding of how to debug with structured log events. Enjoy.

Part 1: you’re reading it :)

Part 2: Build Observable Systems by Sam Stokes

Part 3: Constructing a Coherent Narrative by Colin Curtin

Part 4: Designing for Results by Matt Klein

Part 5: Moar Context Better Events by Mark McBride

Part 6: Thou Shalt Not Aggregate at Source by Alex Rasmussen

Part 7: Building Better Events by Rachel Perkins

Part 8: What Should I Add to an Event? by Ben Hartshorne

 

Related Posts

Customer Stories   Observability  

Customer Story: Intercom Reduces MTTWTF With Observability and Distributed Tracing

Intercom’s mission is to build better communication between businesses and their customers. With that in mind, they began their journey away from metrics alone and...

Observability   OpenTelemetry  

Monitoring Cloud Database Costs with OpenTelemetry and Honeycomb

In the last few years, the usage of databases that charge by request, query, or insert—rather than by provisioned compute infrastructure (e.g., CPU, RAM, etc.)—has...

News & Announcements   Observability  

New Honeycomb Features Raise the Bar for What Observability Should Do for You

As long as humans have written software, we’ve needed to understand why our expectations (the logic we thought we wrote) don’t match reality (the logic...