Ask Miss O11y  

Ask Miss O11y: Pre-production Environments

By Jessica Kerr  |   Last modified on April 20, 2022

Should we build and run the full observability stack in pre-prod? How much realism vs. waiting for prod?

Answer: Yes. You absolutely want observability in pre-production environments—local, dev, performance test, staging, CI, everywhere.

Why?

  • Observability is an essential part of production. Verify your traces when you verify the rest of the system.
  • Test environments are about finding problems early. Which means you’ll need to debug those problems. Which requires observability so you can answer questions like: In an integration test, which system is failing? In a performance test, what is slow? This is observability at work.
  • In development and local environments, iterate on your traces. As part of development, ask “How will I know this works?” and then check whether the traces, spans, and attributes your code emits are giving you that insight.
  • While you’re querying your traces in test environments, notice schema conflicts, like: “Oops, we called it status_code here but everything else uses status.code.” Getting the attributes names right during test means your production schema will be cleaner.

Observability software like Honeycomb emerged from the desperate need of managing microservices and cloud, when older ways of understanding and checking our systems just weren’t enough. We had to create something better.

And observability is better: for instance, distributed tracing with query-time aggregation is strictly more powerful than looking at logs. It is more powerful than looking only at the output of tests. It is more powerful than canned graphs and than all the older techniques in combination.

But what about when it’s technically possible to rely on output and logs, like in low-volume tests? Observability is still better. It’s our best tool and gets our work of debugging broken tests done faster. At the same time, we sharpen the tools and our skills.

Why settle for less?

Do you have a question for Ask Miss O11y? Send us an email or book some time with one of our developer advocates!

 

Related Posts

Ask Miss O11y   Observability  

How Do I Do Availability Checks in Honeycomb?

Let’s dig into what we mean by an Availability Check and how that maps to observability, tracing, and supporting production systems....

Ask Miss O11y   Instrumentation  

Ask Miss O11y: How Can I Convince My Organization to Invest in Instrumenting for Observability?

"Dear Miss O11y, I’ve been following Honeycomb for a long time, and I understand where the insights from observability fit in. But larger orgs haven’t...

Ask Miss O11y   Observability  

Ask Miss O11y: What Should An "Observability Team" Do?

People use “observability team” as a catchall basket for all kinds of things these days—from cutting-edge tech to truly heinous hacks. Eh, it is what...