Software Engineering   Culture  

Engineers New to Honeycomb, What Did You First Notice About How We Do Things Here?

By Honeycomb  |   Last modified on August 2, 2022

We’ve wondered, in the past, what new engineers think about how we do things at Honeycomb. This time, we asked!

Meet Elliott and Reid, two of our engineers that recently hit their 90 day mark. Along with the title question, we also asked about their prior companies, how we differ, and what surprised them most about working here. Let’s get to know them!

Q&A with Elliott and Reid

Q: Let’s get to know you! What’s your name, how long have you been here, and what do you do, as an engineer? Where did you work before?

Elliott: My name is Elliott! I’m a Product Engineer on the Growth team, and I’ve been here for just about 3 months now. Before Honeycomb, I worked at a large company that’s been in business for a while with about 200k total employees. There are many hard problems that show up in organizations that big and old, and not a lot of easy solutions!

Reid: My name is Reid, I just hit my 90 days on the SRE team. I’ve spent most of my time at startups or smaller companies. My first DevOps role was at a startup consulting for other startups. I then moved on (burned out) to internal roles. My last company, a data analytics startup, was probably the biggest I’ve worked at (~500 people) where I split my time between “traditional DevOps” (i.e. cloud sysadmin) and SRE work. I eventually moved onto the Observability team before coming to Honeycomb.

Q: What was the first thing you noticed about working at Honeycomb?

E: The first thing I noticed was how welcoming and friendly everyone was. I felt like I was part of the team right away—and if I’m being honest, I felt that way even during the interviews before I was actually part of the team! The very first time I got a change into production and presented it at the weekly Demo Day meeting, I was blown away by how excited everyone was for even a small styling change.

R: Same, I was also blown away by how pumped people were for demos. At some previous companies, demos were a semi-skippable event where you were unsure of the value of the feature. Here, every little feature is a mini-party because it’s something they, a customer, a prospect, or their team wanted! The energy is infectious.

There’s also a strong debugging culture. We have a tight relationship with AWS, but we don’t throw stuff over the wall—we try very hard to understand the different aspects of our complex infrastructure.

Q: What’s your favorite difference so far?

E: I think my favorite thing about Honeycomb so far has been how open and transparent things are, and how feedback is encouraged, welcomed, and acted on at every stage. There’s also a wonderful culture of documenting everything in living documents, so it’s easier to find things if I have questions.

R: Multiple people have mentioned to me that it feels like they are able to start healing their scars from other tech companies when coming here. I wouldn’t go as far as to say “tech industry therapy,” but Honeycomb leadership’s investment in good management is something I haven’t seen before.

Q: What was the biggest surprise?

E: How quickly we were able to deploy to production. At my last job, because of the tests and verifications required (manual and automated); the difficulty of coordinating releases across teams with wildly different reporting chains; and the many constraints we had from operating with so many different legacy systems and structures, it often took one to two weeks for a new feature to make it from “I finished coding this” to production… on a good week. At Honeycomb? The same process takes maybe an hour. I’m still in shock!

R: An hour at worst! For me, it’s the emphasis on the social aspects of engineering, despite this being a very engineering and product-driven company. Saying “you can’t just prescribe a process to make it true!” doesn’t turn any heads; everyone I’ve talked to has experience with what it takes to build a great organization (and product), from all different perspectives of the industry.

Q: What are some bad habits you had to unlearn from other companies?

E: Programming in an overly-defensive manner is probably the big one! Due to a combination of feature pressure from outside our team, a slow deploy process, ever-shifting requirements, architecture approvals and restrictions, a relatively high turnover rate, hard deadlines for product launches, and wildly different expectations from different areas of the company on what we were supposed to support, we often had to code in a way that took into account every possible situation. 

Obviously that’s not possible, but it was still much easier to change a configuration value than to redeploy our app if we found a bug or a requirement changed at the last second. I’m very much a perfectionist, so coming to a company that values “Fast and close to right is better than perfect” meant I needed to be more confident in getting things out quickly under the idea that I can always fix it if we find an issue—and that we will actually know if there’s an issue.

R: I think my response to incidents is informed by the semi-traumatic experience of being on call for about 15-25 different startups at once very early in my career, with a lot of internal and external pressure. I often needed to make a judgment call between my own sleep, customer dollars, a colleague’s sleep, and/or the ire of management. I did learn how to work hard and quickly when the pager goes off, but at the cost of my natural higher-order decision making and carefulness.
It’s great to have your endorphins rise when you make a split second decision while driving, but not so great when you’re collaborating with others or triaging a potentially complex systems problem. At Honeycomb, our organization takes so much care to learn from the technical and social aspects of incidents, and to use those insights to level-up everyone’s understanding. This, along with a healthy amount of observability instrumentation, increases our ability to diagnose and remediate proactively, instead of reactively. I expect I’ll be getting used to that for a while.

Interested in joining the Hive?

Want to get the newbee 🐝🐝 experience firsthand? We're hiring across many departments (just so happens we have spots for new engineers, too)! See our open positions, learn more about our culture, and apply today.


Related Posts

Software Engineering   Culture  

Staffing Up Your CoPE

Getting the right people working in the CoPE is crucial to success because these change agents must limber up the organization and promote the flexibility...

Software Engineering   Observability  

Navigating Software Engineering Complexity With Observability

In the not-too-distant past, building software was relatively straightforward. The simplicity of LAMP stacks, Rails, and other well-defined web frameworks provided a stable foundation. Issues...

Software Engineering  

Investigating Mysterious Kafka Broker I/O When Using Confluent Tiered Storage

Earlier this year, we upgraded from Confluent Platform 7.0.10 to 7.6.0. While the upgrade went smoothly, there was one thing that was different from previous...