Ask Miss O11y  

Ask Miss O11y: Pre-production Environments

By Jessica Kerr  |   Last modified on April 20, 2022

Should we build and run the full observability stack in pre-prod? How much realism vs. waiting for prod?

Answer: Yes. You absolutely want observability in pre-production environments—local, dev, performance test, staging, CI, everywhere.

Why?

  • Observability is an essential part of production. Verify your traces when you verify the rest of the system.
  • Test environments are about finding problems early. Which means you’ll need to debug those problems. Which requires observability so you can answer questions like: In an integration test, which system is failing? In a performance test, what is slow? This is observability at work.
  • In development and local environments, iterate on your traces. As part of development, ask “How will I know this works?” and then check whether the traces, spans, and attributes your code emits are giving you that insight.
  • While you’re querying your traces in test environments, notice schema conflicts, like: “Oops, we called it status_code here but everything else uses status.code.” Getting the attributes names right during test means your production schema will be cleaner.

Observability software like Honeycomb emerged from the desperate need of managing microservices and cloud, when older ways of understanding and checking our systems just weren’t enough. We had to create something better.

And observability is better: for instance, distributed tracing with query-time aggregation is strictly more powerful than looking at logs. It is more powerful than looking only at the output of tests. It is more powerful than canned graphs and than all the older techniques in combination.

But what about when it’s technically possible to rely on output and logs, like in low-volume tests? Observability is still better. It’s our best tool and gets our work of debugging broken tests done faster. At the same time, we sharpen the tools and our skills.

Why settle for less?

Do you have a question for Ask Miss O11y? Send us an email or book some time with one of our developer advocates!

 

Related Posts

Tracing   Metrics   Ask Miss O11y  

Ask Miss O11y: To Metric or to Trace?

Dear Miss O11y, I remember reading quite interesting opinions from you about usage of metrics and traces in an application. Did you elaborate on those...

Observability   Ask Miss O11y  

Ask Miss O11y: Is There a Beginner’s Guide On How to Add Observability to Your Applications?

Dear Miss O11y, I want to make my microservices more observable. Currently, I only have logs. I’ll add metrics soon, but I’m not really sure...

Ask Miss O11y  

Ask Miss O11y: Error: missing ‘x-honeycomb-dataset’ header

Your API Key (in the x-honeycomb-team header) tells Honeycomb where to put your data. It specifies a team and an environment. Then, Honeycomb figures out...