Training Video

Distributed Tracing in Honeycomb

June 23, 2020



Pierre Tessier [Sales Engineer|Honeycomb]:

Hi. In this video, we’re going to go over how to use distributed tracing inside of Honeycomb. In order to use tracing inside of Honeycomb, we need to make sure Honeycomb understands the field definitions to render the tracing features properly. To do this, you go to the data set definitions section of Honeycomb, find the data set in question, and go to settings. Once inside here, go to the definitions tab. Definition fields are broken down into two categories. Tracing fields and home fields. You do not need to specify every single field in order for tracing to work, but there are a couple that are required. Trace ID, span ID, parent span ID, and name. Other fields here can be used as helpers to provide even more functionality in and around tracing that we’ll show later on. Once you’ve configured these fields, you can now use tracing inside of Honeycomb.

Tracing shows up in a lot of different areas for you to get into. First, from the homepage itself, you can go down towards the bottom and you’ll see recent traces. Here, you’ll click on this button and you’ll be able to go view the trace directly. Another area is from inside of any query or chart. Let’s go inside of the query here for latency. And when we go to it, we have the Results tab, BubbleUp tab, and a Traces tab. Under the Traces tab, you will get a list of the top traces within that query. You can go ahead and also select any of these to get directly into that trace. Really, anywhere we have a trace ID field, including from raw data or from BubbleUp views will allow you to launch directly into the tracing view. And to continue the ability for you to get into the trace that you really want to get into, even from a heat map you can click on any of the blocks and go directly into the trace behind that block.

Now that we see the tracing view itself, let’s talk about what we have here and what we can do. Starting off with the name field, which is the name of the operation, followed by service name itself that was responsible for that operation being called. Next, you’ll have a bar representing the length of time taken out for each span, all of this represented in a waterfall or hierarchal view. Over on the right, we have more information about the selected span, starting with the heat map. This heat map shows us where this span arrives versus all the other spans of the exact same type inside that same query window. Selecting a different span, we can see how that would show different wakes. Below this are different fields associated with that span itself. And just like anywhere else inside of Honeycomb when you’re working in a query, looking at any of the fields you’ll see a three-dot menu option. Clicking on that allows you to continue to interact with your query, further enforcing the core analysis loop inside of Honeycomb.

Now towards the top of the screen, we have a fields option. Under this option, we can add additional fields inside of our view. Perhaps we want to see the database query as well. So we go ahead and select that, and now we have database query inside of our view. Clicking on the header, we have an option to hide or color the base on this value. Doing so will change the color of all the bars inside of the waterfall chart based on the query itself. Now the trace we’re looking at here, it’s fairly simple, dealing with just a few spans. Sometimes we get traces that are a lot more complex. Here’s a trace that is a lot more complex, dealing with dozens of spans, making the navigation a little bit harder to get through. To help with that, Honeycomb will offer a three-dot menu at every single level inside of your trace itself. So you go ahead and click on that and collapse or expand based on a level of question. And once you’ve done that, now we go ahead and explore, zoom in on what we really care about.

Also, inside of Honeycomb, some spans can have markers or events, or even links associated with them. They’re going to be rendered as circle markers inside of the span itself. And you can see the details about each one underneath the detail section. So here we can see the span event details. This is a point in time going to a specific event dealing with Redis. That there is tracing in a nutshell within Honeycomb. I hope you enjoyed the video.

If you see any typos in this text or have any questions, reach out to