Engineering Best Practices

Engineering Best Practices

Engineering Best Practices

We help people observe, debug, and improve their systems.
Dive into our resources to find out more.

 

All Resources  |   Engineering Best Practices

How to Gain In-Depth Observability of Your Kubernetes Clusters

How to Gain In-Depth Observability of Your Kubernetes Clusters

How to Gain In-Depth Observability of Your Kubernetes Clusters

In the breakneck world of modern software development, it’s hard to find teams that don’t use microservices, containers, and Kubernetes to get code out the door faster. But with the speed of deployment and delivery comes a very high degree

Observability Kubernetes Engineering Best Practices

What Is Platform Engineering? A Getting Started Guide

Modern software development can often feel like orchestrated chaos, and with good reason. The processes, tools, and SaaS services are…

Engineering Best Practices

Getting Started with OpenTelemetry

No one ever said observability would be easy, right? But OpenTelemetry is a significant step towards “easier” observability because it’s…

OpenTelemetry Observability Instrumentation Engineering Best Practices

Getting Started With Distributed Tracing

The more complex the software, the more likely it is to break, so the question facing modern DevOps teams today…

Observability Engineering Best Practices Distributed Tracing

How Observability Helps With Incident Response

No one knows what they don’t know. That rather elusive statement is at the heart of one of the biggest…

Observability Incident Response Engineering Best Practices

Understanding High Cardinality and Its Role in Observability

Complex distributed systems generate vast amounts of granular data that is ideal for diagnostics. Most of the time… But it…

Observability High Cardinality Engineering Best Practices

SLOs, SLAs, SLIs: What’s the Difference?

At the end of the day, so much of software development is wrapped up in two basic concepts: happy customers…

SLOs Observability Engineering Best Practices

What Is Observability Engineering?

Although it may seem like something out of fantasy, it is possible to find out what you don’t know you…

Observability Engineering Best Practices

Upleveling Engineering Productivity with Observability

Promote happier and healthier engineering teams, optimize cost and performance, and deliver more innovation by implementing observability. Whether you’re starting…

Observability Engineering Best Practices

De-Risk Your Deployments with Honeycomb & GitHub Actions Deployment Protection…

De-Risk Your Deployments with Honeycomb & GitHub Actions Deployment Protection Rules

Observability Engineering Best Practices CI/CD Pipelines

Cost-Cutting Strategies and Smart Tooling Choices to Maximize Your Vendor…

In this session, Charity Majors and Gergely Orosz provide advice on how to think through cutting costs in a vendor-dependent…

Engineering Best Practices Cloud Costs

Successful Sampling With Refinery

Honeycomb's trace-aware proxy, Refinery, allows you to sample your data to reduce costs while giving you the ability to ensure…

Observability Engineering Best Practices

Platform Engineering Is the Future of Ops

Dig deep into how the future of ops is headed toward platform engineering. You will learn more about the benefits…

Engineering Best Practices

Learn to Leverage Service Maps with Honeycomb

Having a service map is great, but what can you actually do with one? Wouldn’t it be great if you…

Observability Engineering Best Practices

Options for Managing Telemetry Volume At Scale & How Slack…

You’ll learn the pros and cons of different sampling techniques and how to retain granular visibility into your system state.…

OpenTelemetry Observability Engineering Best Practices

Ep. #55, The Dev Side of Observability with Martin Thwaites of Honeycomb

In episode 55 of o11ycast, Charity and Liz speak with Martin Thwaites, a developer advocate at Honeycomb. They discuss the…

Observability Logs Engineering Best Practices

Accelerate CI/CD with Observability

We are joined in this session by Frank Chen, Senior Staff Software Engineer, Slack, who dives into how Slack gained…

Observability Engineering Best Practices CI/CD Pipelines

Changing Practices on Your Team - Observability Driven Development

In this session, we discuss the patterns we’ve observed at Honeycomb that have helped organizations reap the rewards of observability…

Observability Engineering Best Practices

Debugging with the Core Analysis Loop, and What to Buy…

Resolve incidents quickly with the powerful core analysis loop of an observability tool—where anyone on the team can form hypotheses,…

Observability Engineering Best Practices Debugging

How Observability Relates to DevOps, SRE, & Cloud Native

We unpack how observability fits into the landscape of other modern practices, such as DevOps, SRE (Site Reliability Engineering), and…

Observability Engineering Best Practices

Ep. #52, Service Level Objectives with Alex Hidalgo of Nobl9

In episode 52 of o11ycast, Charity and Jess speak with Alex Hidalgo of Nobl9. Alex shares his formative experiences advocating…

SLOs Engineering Best Practices

Ep. #51, Performance Engineering with Henrik Rexed of Dynatrace

In episode 51 of o11ycast, Charity Majors and Jessica Kerr are joined by Henrik Rexed of Dynatrace. This conversation covers…

Observability Instrumentation Engineering Best Practices

Ep. #50, Identifying Weak Spots with Benjamin Wilms of Steadybit

In episode 50 of o11ycast, Charity Majors and Jessica Kerr are joined by Benjamin Wilms of Steadybit. This conversation examines…

Observability Engineering Best Practices

Best Practices Using Honeycomb and Terraform Provider With Hashicorp

Jason Harley, Honeycomb Senior Customer Architect and Hashicorp Ambassador, and Nic Jackson, Principal Developer Advocate at Hashicorp, demonstrate best practices…

Observability Engineering Best Practices

Honeycomb’s O’Reilly Book Observability Engineering - By Charity Majors, Liz…

Authors Charity Majors, Liz Fong-Jones, and George Miranda cut through the marketing hype to concretely demonstrate what observability is, how…

Observability Engineering Best Practices

How Glofox Gives Their Developers Ownership Over Observability

Pierre Vincent and Rob Meaney were able to join forces to establish a culture of observability and give their developers…

Observability High Cardinality Engineering Best Practices

How to Hot-Swap Active Production Services

Download and read the guide to learn how to hot-swap active production services successfully and what pitfalls to avoid.

Engineering Best Practices

Controlling Features in Production: Observability + Feature Flags

We often think about speed and danger as being closely related, but we now have empirical proof that faster, smaller…

Observability Engineering Best Practices

Building Observability in Your CircleCI Deploy

In this talk, you’ll learn how Honeycomb keeps its CircleCI workflow duration at about 10 minutes per build through parallelizing…

Observability Engineering Best Practices CI/CD Pipelines

Building Observability in Your CircleCI Deploy

In this talk, you’ll learn how Honeycomb keeps its CircleCI workflow duration at about 10 minutes per build through parallelizing…

Observability Engineering Best Practices CI/CD Pipelines

Ep. #47, Outcome-based Observability with Gibbs Cullen of Chronosphere

In episode 47 of o11ycast, Liz Fong-Jones speaks with Gibbs Cullen of Chronosphere. Together they compare the roles of product…

Observability Engineering Best Practices

How to Effectively Lead High Performing Engineering Teams in 2022…

RedMonk’s James Governor and Honeycomb’s Charity Majors discuss what it takes to be the best of the best, based on…

Engineering Best Practices

Observability in Sociotechnical Systems: How to Use Telemetry Data to…

Learn how site reliability engineering (SRE) and tooling, and when helpful tools lead us to dark patterns that hurt us,…

Observability Engineering Best Practices

Ep. #44, Examining OpenTelemetry with Vincent Behar of Ubisoft

In episode 44 of o11ycast, Liz and Charity speak with Vincent Behar of Ubisoft. Together they pull back the curtain…

OpenTelemetry Engineering Best Practices

How Intelligent Medical Objects Solved a Mysterious Spike in Latency…

Michael Ericksen, Staff Site Reliability Engineer, explains how IMO implemented observability and established a culture of production excellence when a…

Observability Engineering Best Practices Debugging

How to Use a Single Wide Event to Its Fullest…

Glen Mailer, Senior Staff Software Engineer at CircleCI believes not everything can or should be solved with tracing and sampling.…

Observability Engineering Best Practices

Ep. #43, Going Serverless with Apostolis (Toli) Apostolidis of Cinch

In episode 43 of o11ycast, Charity and Liz speak with Apostolis (Toli) Apostolidis of Cinch. They discuss Cinch's move from…

Engineering Best Practices

Ep. #42, Continuous Profiling with Dmitry Filimonov of Pyroscope

In episode 42 of o11ycast, Charity and Shelby are joined by Dmitry Filimonov of Pyroscope. They discuss continuous profiling practices,…

Engineering Best Practices

Adopting Observability: Lessons Learned On How to Reduce Cognitive Load

At hnycon, we hosted a roundtable discussion with a few of our guest speakers about the lessons they’ve learned while…

Observability Engineering Best Practices

How Refinery Helps With Sampling Complex Event Data

In his talk at the 2021 hnycon, Kevan Carstensen, Backend Engineer at GOAT, explained that GOAT’s small backend team is…

Observability Engineering Best Practices

Coding a New Customer Experience: How Vanguard Upleveled Their Org…

Rich Anakor, Chief Solutions Architect at Vanguard, runs a small team with a colossal goal: Give Vanguard customers a top-notch…

Observability Engineering Best Practices

Four Lessons Red Hat Learned From Refactoring a Monolithic Service…

John Casey is a DevOps engineer and architect who leads the team at Red Hat that’s responsible for CI/CD and…

Migrations Engineering Best Practices

Mode Analytics: Hot-Swapping Active Production Services Reliably With Honeycomb

The Mode Analytics backend team is updating different services in their stack by hot swapping them in production. Watch this…

OpenTelemetry Observability Migrations Engineering Best Practices Customers

Hot Swapping Active Production Services Reliably With Honeycomb

The Mode Analytics backend team is updating different services in their stack by hot swapping them in production. Watch this…

OpenTelemetry Observability Migrations Engineering Best Practices

Ep. #41, Organizing Conferences with Aaron Aldrich of Red Hat

In episode 41 of o11ycast, Charity and Liz speak with Aaron Aldrich of Red Hat. Together they pull back the…

Engineering Best Practices Debugging

Vanguard's Adoption Journey: How Honeycomb Helps Shape Developer Workflows

After evaluating multiple approaches to distributed tracing, Vanguard ultimately landed on using OpenTelemetry and Honeycomb. Now, they have hundreds of…

OpenTelemetry Observability Engineering Best Practices Customers

CircleCI: The Unreasonable Effectiveness of a Single Wide Event

Using only a single event for each valuable business transaction, and packing it with fields, we can reduce event volume…

Observability Engineering Best Practices

Stepping Our Way Into Resilient Services

Is it possible to discover unknown-unknowns proactively with Chaos Engineering? Where exactly is the intersection between intentionally breaking production services…

Observability Engineering Best Practices

Driving Observability Adoption Forward as an Internal Champion... During a…

Josh from Amperity covers how they planned, strategized, and organized their observability tooling evaluation. Josh also shares how the COVID-19…

Observability Engineering Best Practices

Killing Your Staging Environment

“But it worked in staging!” is the new “But it works in my machine!” Docker, Kubernetes, and friends have made…

Engineering Best Practices

Ep. #40, Player Experience with Nick Herring of CCP Games

In episode 40 of o11ycast, Liz and Charity speak with Nick Herring of CCP Games. They discuss the challenges faced…

Observability Engineering Best Practices Debugging

Ep. #37, Synchronization Layers and Logs with Garry Shutler of…

In episode 37 of o11ycast, Charity and Shelby speak with Garry Shutler of Cronofy. Together they pull back the curtain…

Observability Engineering Best Practices

Ep. #36, Resilience Engineering with Jacob Scott of Stripe

In episode 36 of o11ycast, Charity and Liz speak with Jacob Scott of Stripe about the need for SRE teams,…

Engineering Best Practices Distributed Tracing

DevOps Unbound- The Road to GitOps

Hosts Alan Shimel and Mitch Ashley are joined by Kurt Chase ‎Senior Director, Global Release Management of Tricentis, Charity Majors,…

Observability Engineering Best Practices

Ep. #35, Government Software Systems with Mark Ferlatte of Truss

In episode 35 of o11ycast, Charity and Shelby speak with Mark Ferlatte of Truss. They discuss government software system failures…

Engineering Best Practices

Ep. #33, Information Accessibility with Katy Farmer of CircleCI

In episode 33 of o11ycast, Charity and Shelby are joined by Katy Farmer of CircleCI. They discuss learned helplessness, understanding…

Engineering Best Practices CI/CD Pipelines

Habits of Highly-Performing Teams

There is a yawning gap opening up between the best and the rest—the elite top few percent of engineering teams…

Engineering Best Practices

Ep. #32, Managing Hardware with Gianluca Arbezzano of Equinix Metal

In episode 32 of o11ycast, Liz and Shelby speak with Gianluca Arbezzano of Equinix Metal. They discuss diversifying the way…

Observability Engineering Best Practices

Ep. #31, Rewarding Curiosity with Ellen Chisa of Dark

In episode 31 of o11ycast, Charity Majors and Ellen Chisa of Dark discuss the benefits of making observability tools more…

Observability Engineering Best Practices

Ep. #30, Harder Conversations with Beau Lyddon of Workday

In episode 30 of o11ycast, Charity and Liz speak with Beau Lyddon of Workday about the recent sea change in…

Metrics Engineering Best Practices

Ep. #29, Testing in Production with Glen Mailer of CircleCI

In episode 29 of o11ycast, Charity and Shelby are joined by Glen Mailer of CircleCI. They discuss testing in production…

Engineering Best Practices Debugging CI/CD Pipelines

Ep. #28, People Problems with Austin Parker of Lightstep

In episode 28 of o11ycast, Charity and Shelby speak with Austin Parker of Lightstep. They discuss rethinking human error, purposeful…

Engineering Best Practices

Identifying Hidden Dependencies

Learn how Honeycomb improved the reliability of our Zookeeper, Kafka, and stateful storage systems through terminating nodes on purpose.

Engineering Best Practices AWS

Embedding Observability Into Your Engineering Culture

How can you create a safe culture that enables engineers to learn, try, and test? The data unlocked by observability…

Observability Engineering Best Practices

Cultivating Production Excellence

Taming the complex distributed systems is not just changing tools and techniques. It also requires changing who is involved in…

Engineering Best Practices

Fast & Simple: Observing Code & Infra Deployments at Honeycomb,…

In this talk, you'll learn about the architecture and operational practices used by an engineering team of less than a…

Observability Engineering Best Practices

Progressive Delivery: Using feature flags & observability to ship confidently

Progressive delivery lets you see whether or not new changes will benefit users before they’re really released. This webinar explores…

Engineering Best Practices Customers CI/CD Pipelines

Progressive Delivery: Using Feature Flags & Observability to Ship Confidently

Progressive delivery lets you see whether or not new changes will benefit users before they’re really released. This webinar explores…

Observability Engineering Best Practices CI/CD Pipelines

Ep. #25, Reliability First with Amy Tobey of Blameless

In episode 25 of O11ycast, Charity and Shelby speak with Amy Tobey of Blameless. They explore the evolution of the…

Incident Response Engineering Best Practices

Ep. #24, Beyond Code with Heidi Waterhouse of LaunchDarkly

In episode 24 of O11ycast, Charity and Liz speak with Heidi Waterhouse of LaunchDarkly. They discuss feature flags, value stream…

Engineering Best Practices

Ask an SRE: Panel Discussion

Follow along as SRE leaders from Slack, Fastly, Catchpoint, and Autodesk discuss defining SRE, avoiding pitfalls, managing incident anxiety, informing…

Engineering Best Practices

Ep. #23, Beyond Ops with Erwin van der Koogh of…

In episode 23 of O11ycast, Charity and Shelby speak with Erwin van der Koogh of Linc. They discuss the startup…

Engineering Best Practices

Ep. #22, Designing for Observability with Jimmy Bogard of Headspring

In episode 22 of O11ycast, Liz and Charity speak with Jimmy Bogard of Headspring. They discuss maintaining balance for on-call…

Migrations Incident Response Engineering Best Practices

Ep. #21, Learning Systems with Jessica Kerr

In episode 21 of O11ycast, Charity and Liz speak with Jessica Kerr. Together, they explore complex learning systems, how to…

Engineering Best Practices

Ep. #20, Between Dev and Ops with Marco Rogers of Mode

In episode 20 of o11ycast, Charity and Shelby speak with Marco Rogers of Mode. They discuss the wall between dev…

Engineering Best Practices

The Socio-Technical Path to High Performing Teams - With Animation

We are far behind where we should be as a profession when it comes to how much of our effort…

Observability Engineering Best Practices

Ep #19, Giving Observability a Name with Shelby Spees of Honeycomb

In episode 19 of O11ycast, Liz and Charity speak with Shelby Spees of Honeycomb. They discuss Shelby’s diverse engineering background,…

Observability Engineering Best Practices

Tradeoffs on the Road to Observability - With Animation

The main way to improve sustainability of services is to empower engineers, not just SREs, to understand their systems. Liz…

Observability Engineering Best Practices

Ep. #17, Leveling Up Software Delivery with Pete Hodgson

In episode 17 of O11ycast, Charity and Liz are joined by Pete Hodgson, an independent software delivery consultant. They discuss…

Observability Engineering Best Practices CI/CD Pipelines

SEDNA’s Product Team Reduces Reliance on Internal Team Knowledge with…

SEDNA's Product team wanted to consolidate and bring consistency to their monitoring and alerting. Incident response relied too much on…

Observability High Cardinality Engineering Best Practices Customers

Ep. #15, Speaking of Serverless with Rachel Myers and Emily Nakashima

In episode 15 of O11ycast, Charity and Liz speak with Rachel Myers of Google and Emily Nakashima of Honeycomb. They…

Logs Engineering Best Practices

Eaze Into Observability

Join us to learn how Eaze reduced the costs of running their service and increased customer happiness at the same…

Observability Logs Engineering Best Practices Customers

Ep. #14, Team Players with Mehdi Daoudi of Catchpoint

In episode 14 of O11ycast, Charity and Liz are joined by Mehdi Daoudi of Catchpoint. They discuss the importance of…

Engineering Best Practices

Ep. #13, Cloud Wrangling with Natalie Bennett of Pivotal

In episode 13 of O11ycast, Charity Majors and Liz Fong-Jones talk with Natalie Bennett, Software Engineering Manager at Pivotal. They…

Engineering Best Practices CI/CD Pipelines

Ep. #12, Speed of Deployment with Rich Archbold of Intercom

In episode 12 of O11ycast, Charity Majors and Liz Fong-Jones speak with Rich Archbold of Intercom. They discuss the crucial…

Observability High Cardinality Engineering Best Practices

Ep. #11, Chaos Engineering with Ana Medina of Gremlin

In episode 11 of O11ycast, Charity Majors and Liz Fong-Jones speak with Gremlin chaos engineer Ana Medina. They discuss the…

Engineering Best Practices

Ep. #10, Measuring Technical Debt with Bugsnag CEO James Smith

In episode 10 of O11ycast, Charity Majors and Liz Fong-Jones speak with Bugsnag CEO James Smith. They discuss the seemingly…

Engineering Best Practices

Developing a Culture of Observability

Observability gives engineers insight into how their systems function and how users experience the resulting services; it allows you to…

Observability Engineering Best Practices

Ep. #9, High Performance DevOps with Jez Humble

In episode 9 of o11ycast, Charity and Rachel sit down with Jez Humble, Co-Founder and CTO of DevOps Research and…

Observability Engineering Best Practices

Ep. #8, Email Ecosystems & APIs with Nylas' Christine Spang

In episode 8 of O11ycast, Charity and Rachel are joined by Nylas Co-Founder and CTO Christine Spang to discuss navigating…

Engineering Best Practices

Ep. #6, Customer Reliability Engineering with Google's Liz Fong-Jones

In episode 6 of O11ycast, Charity and Rachel talk to Google developer advocate Liz Fong-Jones about ways to build systems…

Engineering Best Practices

Ep. #5, InfoSec with Gartner's Anton Chuvakin

In episode 5 of O11ycast, Rachel and Charity speak with Anton Chuvakin, Research VP at Gartner for Technical Professionals, about…

Engineering Best Practices

Ep. #4, Systems Administration with Adam Jacob of Chef

In episode 4 of O11ycast, Rachel and Charity ask Adam Jacob, co-founder and CTO of Chef, how the culture of…

Engineering Best Practices

Ep. #3, Distributed Systems with Paul Biggar of Dark

In episode three of O11ycast, Rachel and Charity are joined by Paul Biggar, co-founder and CTO of Dark as well…

Engineering Best Practices

Ep. #2, Observable DevOps with Christina Noren of CloudBees

In episode two of O11ycast, Rachel and Charity are joined by Christina Noren, Chief Product Officer for CloudBees, to look…

Observability Engineering Best Practices
Scroll to Top