Automatic Instrumentation for OpenTelemetry Go

The OpenTelemetry Go project now supports automatic instrumentation via eBPF! This is a big milestone for the project and makes it significantly easier to generate data from your Go apps:

• The agent is a drop-in for any running application, meaning you don’t need to redeploy to get data
• The agent automatically detects libraries you’re using and instruments them
• The agent supports net/http, gRPC, gin, and MUX instrumentation today

The automatic instrumentation agent is still in s/alpha/beta today, but it’s ready for you to try on your applications!

Getting started

You can use the OpenTelemetry Go agent on bare metal or in Kubernetes. Check out our getting started docs that describe how to configure it to run in either environment.

Using eBPF

As mentioned, the agent uses eBPF to generate instrumentation. Let’s elaborate on that briefly.

eBPF is a capability in OS kernels that lets you run sandboxed programs, known as “probes.” These probes can efficiently gather information from the underlying OS in several ways. In particular, there are different types of probes, each suitable for different kinds of tasks. The automatic instrumentation agent uses uprobes and uretprobes to inspect running Go programs and create spans for calls to supported libraries (like net/http).

So, why is this necessary? Languages like Java or JS compile down to a bytecode that gets interpreted by a language runtime. That bytecode is capable of being “patched” with instrumentation calls, which allows for a relatively straightforward way to automatically instrument applications. This is typically done with an agent that runs side by side with the application it’s instrumenting.

Go, like C++ and Rust, compiles directly into machine code. It’s not “patchable” like other languages, so the techniques for automatic instrumentation can’t be used. Instead, you need something that can inspect running programs on an OS. Luckily, that’s what projects such as eBPF allow you to do! And in fact, this is an explicit goal of eBPF: to allow for this kind of inspection at runtime in a completely language-agnostic way.

By using eBPF, the automatic instrumentation agent for Go can analyze when a supported call is made, then instrument that call.

What’s next for Go, eBPF, and Honeycomb

The OpenTelemetry Go project will continue to improve support for libraries to automatically instrument. As contributors and co-maintainers, Honeycomb is keen on improving the agent so that all our customers can use it.

For eBPF more generally, we’re exploring more ways to use eBPF to automatically instrument any application and create an easy-to-use onboarding tool without having to modify code or install something that’s language-specific. Stay tuned to learn more, and in the meantime, sign up for Honeycomb today!