HAProxy Observability

HAProxy is often the first point of contact for interaction with your service. With a very small amount of instrumentation, we can reliably find the source of any slowness in a massive distributed system in just a few clicks.

By consuming HAProxy logs, Honeycomb makes it possible to answer deep questions about systems instantly.

haproxy observability
haproxy observability - log config

Data Collection

The honeytail agent captures and streams log files to Honeycomb as they’re written. Honeytail converts the strings into structured json objects and reads the config file to add any extra fields defined, so we don’t have to change any regular expressions (or other hacks) to detect the new information.

You can also backfill old logs into Honeycomb to look at past data.

Getting Answers

When debugging a complex system, we usually want to start at the edge and systematically work our way down. With HAProxy logs in Honeycomb, you can ask questions like:

  • Which endpoints are slowest?
  • Is the increased latency distributed evenly across endpoints, or only some?
  • If only some endpoints are slower than usual, are there outliers inflating the average?
  • What do these outliers have in common? Are we being DDOSed?
  • How much time are your queries spending in a queue waiting for a backend server?
  • Which backend server pools have too much load and are getting queries stuck in queues waiting to be handled?
honeycomb.io query chart example