The New Rules of Sampling
It is not feasible to run an observability infrastructure that is the same size as your production infrastructure. Past a certain scale, the cost to collect, process, and save every log entry, every event, and every trace that your systems generate dramatically outweighs the benefits. The question is, how to scale back the flood of data without losing the crucial information your engineering team needs to troubleshoot and understand your system's production behaviors? Sampling is the key.
Geckoboard Gets the Context They Need to Keep Customers Happy Without Breaking the Bank
Geckoboard built their own system health dashboards and have been using standard log search tools since the time when log search was new and shiny. But as their business has scaled up, so has their need to understand specifically who is affected by a given issue.