Use Derived Columns To Prioritize Development Work


We recently released a new feature for Honeycomb: derived columns and we promised at the time that we’d show you some more examples of how it can make your life easier. Here are a couple that are about helping you figure out wtf you should do next:

What SDK(s) should we work on the most/next?

We’ve got so much to do, but we can’t do it all at once. You know how it is. Sometimes, you’ve got to prioritize things.

Using a derived column, we can look at the contents of user_agent from data our customers send us and use it to figure out which SDK most people are using. If there’s a definite winner or winners, we should most likely tackle those first when adding features or fixing bugs.

(using the derived column sdk = REG_VALUE($user_agent, "libhoney-[a-z]*"))

And here’s the resulting output:

Oh, hm–those two Manticore entries are not really what we want here–Manticore is a Ruby HTTP library that’s used internally by our logstash plugin, but we don’t develop it, so let’s filter out anything that’s not really an SDK:

Ah, much better. (Aside: whoah, what happened with that libhoney-go/1.3.0 version?) In any case, libhoney-go/1.3.3 is the clear winner.

When can we safely deprecate support of old API versions?

Older versions of APIs can be slow or buggy, so we do what we can to encourage our users to upgrade to the latest versions of our logstash plugins (and stop using the associated out-of-date API). However, they don’t necessarily write to tell us when they do (jerx! ;)). Have no fear, though–we can use a derived column to check and see if anyone is still left behind and if not–begin the process of decommissioning that old plugin.

(using the derived column logstash_plugin_version = REG_VALUE($extra_headers, "X-Plugin-Version:[0-9\.]*"))

And the resulting output:

(Note: The unnamed plugin at the bottom is all the traffic that doesn’t come from logstash.)

Hmm. Doesn’t look as though most people have upgraded to the new version yet, so we can’t really deprecate the old one. Guess we need to spend a little more time reminding people to try the new version…