Ask Miss O11y  

Ask Miss O11y: Pre-production Environments

By Jessica Kerr   |   January 28, 2022

Should we build and run the full observability stack in pre-prod? How much realism vs. waiting for prod?

Answer: Yes. You absolutely want observability in pre-production environments—local, dev, performance test, staging, CI, everywhere.

Why?

  • Observability is an essential part of production. Verify your traces when you verify the rest of the system.
  • Test environments are about finding problems early. Which means you’ll need to debug those problems. Which requires observability so you can answer questions like: In an integration test, which system is failing? In a performance test, what is slow? This is observability at work.
  • In development and local environments, iterate on your traces. As part of development, ask “How will I know this works?” and then check whether the traces, spans, and attributes your code emits are giving you that insight.
  • While you’re querying your traces in test environments, notice schema conflicts, like: “Oops, we called it status_code here but everything else uses status.code.” Getting the attributes names right during test means your production schema will be cleaner.

Observability software like Honeycomb emerged from the desperate need of managing microservices and cloud, when older ways of understanding and checking our systems just weren’t enough. We had to create something better.

And observability is better: for instance, distributed tracing with query-time aggregation is strictly more powerful than looking at logs. It is more powerful than looking only at the output of tests. It is more powerful than canned graphs and than all the older techniques in combination.

But what about when it’s technically possible to rely on output and logs, like in low-volume tests? Observability is still better. It’s our best tool and gets our work of debugging broken tests done faster. At the same time, we sharpen the tools and our skills.

Why settle for less?

Do you have a question for Ask Miss O11y? Send us an email or book some time with one of our developer advocates!

 

Related Posts

Ask Miss O11y  

Ask Miss O11y: Is the OpenTelemetry Collector useful?

Dear Miss O11y, I’ve been told I need to use the OpenTelemetry Collector, but I have no idea what it is, or why I need...

Ask Miss O11y   Tracing  

Ask Miss O11y: Not Your Aunt’s Tracing

Dear Miss O11y, How is modern observability using tracing, such as Honeycomb, different from the previous distributed tracing software I'm familiar with, like Dapper, at...

Ask Miss O11y   Instrumentation  

Ask Miss O11y: When should you delete instrumentation? 

When do you delete instrumentation? You delete instrumentation when you delete code. Other than that, if you’re doing things right: almost never. One of the...