Ask Miss O11y: Making Sense of OpenTelemetry—ContextBy Jessica Kerr | Last modified on September 14, 2022
“What is up with the Context in OpenTelemetry? Why do I need to mess with it at all? Why, when I set a span as active, don’t subsequent spans just use it as a parent?”
Oh, yikes, yeah. The Context abstraction in OpenTelemetry is hard to understand. In fact, there are several ways it’s tricky.
What is an OpenTelemetry Context?
A Context is roughly “you are here.” Say you want to create a new span—a Context says where that span fits. Does this new span start a new trace? Or does this new span have a parent, and what is that parent span? The Context has this information. In OpenTelemetry, there’s always an active context, even if it is empty. You are always somewhere.
What is OpenTelemetry Context Propagation?
For distributed traces, the Context follows the flow of execution across network calls. It gets encoded into HTTP headers, grpc metadata, or somehow added to the message. The receiving service picks that information up and establishes its Context.
What kind of object is the Context?
- It’s a key-value store. That’s all. What’s so special about that?
ActivitySource. In Go, there’s already a
context. In Java, there’s a
- It’s often implicit, but then other times we have to use it very explicitly.
- It’s immutable, so we can’t change it, yet it seems to change on us.
Tracing Context with spans in OpenTelemetry
What’s special about the Context is not the store itself. Indeed, it holds keys and values, like any old dictionary.
What is special is how it follows the thread of execution. The Context carries a “you are here” pointer, so that when we create spans with the active context, those spans link up with a parent span in a trace. The trace becomes a map of a full request’s processing.
The mechanism for this varies by language. In Java, there’s
ThreadLocal storage. In Node.js,
AsyncLocalStorage. In .NET,
System.Diagnostics. In Go, there’s no sneaky way, so context is passed around in explicitly, using existing conventions.
In each of these ways, the active context is available to OpenTelemetry libraries, so that in our code we can easily create spans, like with
Then if we want our span to ever be a parent to another span, suddenly pulling a context out of the air isn’t enough. We need to change the context, so that future spans use ours as a parent.
But you can’t change the context! It’s immutable. It’s gotta be immutable, because other code that runs concurrently might still be using this Context.
So instead, we have to create a new Context, one that thinks our span is the current span. If we want to create our own child spans of this context, we can hold onto it and pass it when creating those spans.
That works for spans we create, but what about spans created by libraries? What if I want my span, called “call a bunch of other services,” to be the parent of the spans automatically created every time I make an HTTP request?
The HTTP-call instrumentation is going to pull the active Context out of thin air. I need to get my Context, which knows about my valuable parent span, into the thin air.
To do this, you need to wrap the call to whatever function or block of code represents the work of this span. So I want to execute my
callABunchOfOtherServices code inside a Context that knows about my span.
In Java, that works out to a try-with-resources block using
context.with()(or for a shortcut, to
startActiveSpan). These calls modify whatever magic storage mechanism the language provides–only for the code I delineated, not for anything else running in the Context my code started with.
This is the pattern for making your own spans that integrate with auto-instrumented spans. Pull the active context out of the air → create a span with that → create a new context that knows about that span → execute more code inside that context. Any spans created by that code will fit into the trace as children.
Ask Miss O11y for Further Help With OpenTelemetry Context
Distributed tracing is a kind of magic. It weaves together a story out of each span’s “you are here” markers. That story is powerful when you want to understand causality in your software.
"Dear Miss O11y, I’ve been following Honeycomb for a long time, and I understand where the insights from observability fit in. But larger orgs haven’t...
People use “observability team” as a catchall basket for all kinds of things these days—from cutting-edge tech to truly heinous hacks. Eh, it is what...