I want to know what users are doing in my application. A distributed trace is the best way to show the data flow of one user interaction through my application, but it isn’t sufficient to show the overall user experience. 

Kubernetes is widely used for deploying, scaling, and managing systems and applications and is an industry standard for container orchestration. Google engineers originally developed Kubernetes as an open-source project. Its first release was in September 2014, and since then, it has matured into a graduate project maintained by the Cloud Native Computing Foundation (CNCF). With the complexities of scale and distributed systems, debugging in Kubernetes environments can be difficult. 

In this post, I’ll cover what synthetic monitoring is and show an example of how you can create a simple monitor using OpenTelemetry, .NET, and an Azure function. If you only want to see how it’s built, skip ahead to building a synthetic monitor.

In last week’s piece, we talked about some of the factors that are driving costs up, both good and bad, and about whether your observability bill is (or should be) more of a cost center or an investment. In this piece, I’m going to talk more in depth about cost drivers and levers of control.

The session.id field is special in Honeycomb for Frontend Observability. It’s a default option for filtering and grouping, and it’s the basis for session timeline analysis (in Early Access). Now you can control how session.id is set.

The idea that telemetry data needs to be managed, or needs a strategy, draws a lot of inspiration from the data world (as in, BI and Data Engineering). Your company most likely has a data team that manages the data warehouse(s), data pipelines, data sources, and reporting tools. These teams are also constantly balancing costs with their user and stakeholder needs, usability, data retention, granularity, etc. Sound familiar? That’s because if you’re working on observability data, these teams are at least several years ahead of you in addressing these tradeoffs and considerations—and can teach us quite a lot. 

In 2018, I dashed off a punchy little blog post in which I observed that teams with good observability seemed to spend around ~20-30% of their infra bill to get it. I also noted this was based on absolutely no data, only my own experiences and a bunch of anecdotes, heavily weighted towards startups and the mid-market tech sector.

Seems like you can’t throw a rock without hitting an announcement about a Model Context Protocol server release from your favorite application or developer tool. While I could just write a couple hundred words about the Honeycomb MCP server, I’d rather walk you through the experience of building it, some of the challenges and successes we’ve seen while building and using it, and talk through what’s next. It should be pretty exciting, so strap in!

We’re excited to share that Honeycomb has completed our first-ever acquisition: we’re joining forces with Grit, bringing aboard not only a strong team but also compelling technology that supercharges our ability to deliver on our mission: to bring observability to every software engineer.

Observability is way more about software engineering than it is about operations. Operators are users of observability data for monitoring and alerting on systems. They’ll use that telemetry data to scale systems, or potentially debug the outside of applications. In contrast, software engineers are creators, designers, and users of observability data, and where they use that data is much wider than that of pure operators of production systems.

You can’t throw a rock without hitting an online discussion about ‘vibe coding,’ so I figured I’d add some signal to the noise and discuss how I’ve been using AI-driven coding tools with observability platforms like Honeycomb over the past six months. This isn’t an exhaustive guide, and not everything I say is going to be useful to everyone—but hopefully it will clear up some common misconceptions and help folks out.