The Raw & Real Approach to Observability


Practicing observability isn’t just about tools. It also means improving how you work together and how you share lessons across the team. Learning from each other helps everyone on your team become better engineers that can create amazing experiences with code, or that make code work at incredible scale (or both!). Writing software and operating it in production is—and must be—a team sport.

What we often hear at Honeycomb is that many folks would love to know what particular techniques we use to actualize productive teamwork and help each other work in more successful ways.

With that in mind, we invite you to participate in our newest series: Raw & Real.

Walking the walk

In unpacking the topic of observability, people sometimes talk about how things should be. It’s important to discuss the ways that teams should work together because that’s how we create a vision that we can strive toward together.

Where the rubber meets the road is taking that vision and applying it to teams made of unique individuals, with differing dynamics, cultures, and experiences. Are we all supposed to do the same things? How do other people do it? When adopting new approaches, it helps to see real-world examples of what other teams are doing and how that changes the ways they work.

What does this actually look like in practice? It’s one thing to talk the talk. But how does Honeycomb walk the walk?

Raw & Real is a short-form live video series that exists to show you how Honeycomb uses Honeycomb. You know that Honeycomb ingests all of your events and helps you analyze them. What’s not always apparent is that Honeycomb also ingests and helps analyze everyone else’s events, along with our own. Meaning, we use Honeycomb to guide how we write and operate Honeycomb in production. We dogfood the dogfood you’re dogfooding.

A dog sitting at a table with head tilted, gazing at a line of dog treats
At Honeycomb, we eat our own dogfood.

Raw & Real is informative, short, and to the point. We sit down and have an informal chat about one key aspect of using Honeycomb that might help you uncover new ways to use Honeycomb for yourself. It’s not a presentation. It’s not a webinar. There’s no script. It’s just raw and real conversations about how we use Honeycomb to help ourselves, and each other, become better engineers.

The episodes are live virtual events about once a month. We start with a small specific lesson, followed by a conversation to answer your questions. It’s a bit like group therapy virtual office hours focused on one specific topic. Drop in and learn a little bit about how we do things.

Past discussion topics

What kinds of topics do we cover?

Episode 1 shows you what it’s like to be a mind reader (almost!) by using Honeycomb’s activity feed. It’s not just a history feed. It helps you drill down to see the method of investigation that helped your coworkers find that ever-elusive needle in the haystack. It’s like learning how to troubleshoot through someone else’s eyes.

In episode 2, we focus on alleviating alert fatigue by cutting down the noise when you’re on-call. Using Service Level Objectives (SLO’s) helps redefine what actually matters and what alerts people really need to receive. Do you care that some transient pod had a CPU spike? Or is the performance your customers experience the measure that really matters? Focus on better customer experiences AND get paged less. That’s what we all deserve.

For episode 3, we dive deeper to show you how we investigate tough problems, like locating sources of latency. Diagnosing slow performance in distributed systems is hard because, typically, when one thing is slow then everything is slow. How do you even know where to start looking? Come see how we use BubbleUp to make diagnosing hidden issues much easier.

Join our next show!

Our next installment, Episode 4: Build better builds, is on June 24th. We’ll be chatting about how we understand exactly what’s happening in our builds by tracing events, optimizing around bottlenecks, and using markers with queries to show us what’s happening and when it happens. Raw & Real is lots of fun, but it’s even better with YOU in the audience. Come join us live and participate in the discussion.

Keep it real, folks!