Honeycomb Blog

Introducing Honeycomb’s TCP Agent for MongoDB

We’re excited to release honeycomb-tcpagent, an efficient way to get query-level visibility into your MongoDB deployment. honeycomb-tcpagent parses TCP traffic between MongoDB clients and servers, and reconstructs queries in a friendly JSON format. Honeycomb helps you explore this data to quickly uncover anomalies. Get started with Honeycomb and run the agent, or keep reading for more background and examples. Are you running a database that’s not MongoDB? Let us know! Support for MySQL is already in the works. Database Observability Means Lots of Questions About Lots of Data For any serious database performance work, the ability to fully capture a…

Read More...

Instrumentation: Instrumenting HTTP Services

Welcome to the second week of our blog post series on instrumentation, curated by Julia and Charity. This week will focus more on operational and practical examples; check out previous entries for awesome posts on Finite State Machines, The First Four Things You Measure, and more! Instrumenting HTTP Services I spend most of my time at VividCortex working with and building tools for database instrumentation. We’re a SaaS platform with lots of HTTP services, so I spend time thinking about HTTP instrumentation too. Here’s my approach. Services have three sections that we need to instrument. There’s the point where requests…

Read More...

Honeycomb Signups Are Back! (With New & Improved Storage Support)

A long time ago, in a galaxy far far away, some of you may remember signing up for Honeycomb access. (Okay, it was probably sometime in August.) We were astonished by the response, by the hundreds of people who signed up following our initial tweet. We approved about half of the requests we received in August, and then realized we had made some miscalculations about what people would value. So we paused signups. For the past six weeks, we stopped processing new users, and worked instead on getting you to the awesome parts faster and providing unique value. 1) Getting…

Read More...

The MySQL Slow Query Log, AWS Golang SDK, RDS and You

Did you know you can do fun things with the MySQL slow query log, using the nothing but the AWS golang SDK for RDS? It’s true! In order to let you submit RDS logs directly to Honeycomb, we’ve been exploring how to use the golang RDS SDK to tail the mysql slow query log. The things we’ve discovered that we didn’t find in the docs are fascinating. (All of it may very well be there somewhere, but if it is we failed to find it.) For example: The docs say the logs rotate every hour, but they don’t say it’s…

Read More...

MySQL and Honeycomb: My First 10 Minutes

As part of the process of building our RDS connector for Honeycomb, we ran it on our own database logs. A few neat things came out of the first 10 minutes of looking at the graphs. Our most frequent queries all come from the API server (rather than the UI or other jobs). This makes sense, as the API receives a high sustained rate of events. We have some caching for these queries, and we can actually tell that the caching is working based on the periodic queries run on cache expiration. For example, if we dive into a specific…

Read More...

MongoDB and Honeycomb: My First 10 Minutes

Real-world examples An interesting side effect of the devopsification of the software industry is how many software engineers and operations engineers increasingly find ourselves in the role of “Accidental DBA,” responsible for operating, debugging, and maintaining a variety of storage solutions at scale. (Noobs may be interested in Maslow’s Hierarchy of Database Needs, a talk I gave last year on what you should care about and think about as an accidental DBA from the systems or software worlds.) This can be terrifying at times, but also terrifically fun. 🙂 Databases are complex and opaque and can be challenging to debug….

Read More...

How Honeycomb Uses Honeycomb, Part 1: The Long Tail

This post is the first in our series of “dogfooding” posts. At Honeycomb, we dogfood everything: we try to solve all our own problems using our own service. And like any young company, we’re moving fast and uncovering plenty of new problems all the time. We built Honeycomb because we were dissatisfied with the “split brain” flow of being alerted by one dashboard product and falling through to a second log aggregation product for investigation. We wanted to be able to capture transient state in the moment of something happening in our system for later exploration. Here’s an example of…

Read More...