In our world of distributed systems, state changes to your infrastructure often take some time to propagate. With a few exceptions (for example, feature flags), single point in time changes are rare. Deploys, outages, database migrations, failovers, stress tests; none of these things are instantaneous – all have some duration during which the system is changing.
When laying down a Marker on your graph to indicate the state change, it’s useful to not only see the start time but visualize the entire duration overlaying your regular traffic.
The old way wasn’t enough
We’ve always noted both the start and end time in our own Markers when running our internal deploy process, but the Honeycomb UI rendered them as a single point in time:
So we made it better
Recently, we decided to improve the Marker feature so that it actually renders the range:
And it’s amazing. I never want to see a point Marker again. Ranges let us see traffic switching to the new build during the deploy. If the deploy included performance work, we can see the P95(duration) drop during the deploy down to the new level.
How do I get that?
Range Markers are now available for everyone to use, and the Marker documentation is updated.
You can get started using them in a few ways:
Add Markers via the CLI
honeymarker CLI supports both start (
-s) and end (
-e) time flags when creating Markers:
$ honeymarker -k YOUR_WRITE_KEY -d ingest-dataset add \ -m "2 hour ingest outage" \ -s $(ruby -e "p Time.now.to_i-2*60*60") \ -e $(ruby -e "p Time.now.to_i")
and you can also update a Marker with an end time after the fact:
$ honeymarker -k YOUR_WRITE_KEY -d ingest-dataset add \ -m "us-east-1 outage" \ -s $(ruby -e "p Time.now.to_i") # note the ID in the output # ... # and then after the outage resolves $ honeymarker -k YOUR_WRITE_KEY -d ingest-dataset update \ -id ID_FROM_ABOVE \ -e $(ruby -e "p Time.now.to_i")
Refer to the documentation for the complete
honeymarker installation/usage instructions.
Add Markers directly from the UI
You can also create Ranged Markers directly from the UI after selecting the time range:
Integrate directly with the Marker API
Finally, for integration with your deploy scripts, CI, and other tooling, you can always hit the Marker API directly.
Like what you see?
Give it a whirl and show us how you use ranged markers to improve your visibility into your systems–we might send you an “I Test in Prod” t-shirt 🙂
If you’re ready to get started with Honeycomb, sign up for a demo today!