Lattice Watch: Smarter Guardrails for Design System Observability

One of the hardest challenges facing platform teams is wrangling the rising volume of PRs looking to add drift to the systems we've invested in. It's impossible to catch them all, so it's more important than ever to invest in building stronger guardrails so our product teams can keep building quickly and catch issues before they merge to main.

Linters are a great tool to reach for first. They are cheap, they run quickly, and you can set them up in CI to block a merge, which guarantees developers will listen to them. In token-based design systems, you can easily add linters for all your token sets, and that gets you strong guardrails against hardcoded values. But linters have a weak spot: they're rigid, and real code needs flexibility.

It's common for developers to occasionally deviate from a system, but in today's world, with more and more agents coding, it's become an issue because agents will do almost anything to get your UI looking how you want. Linters are great for catching obvious drift, but they lack the capacity to reason about a creative bypass or determine if a specific deviation is truly warranted. This type of high-context, nuanced evaluation previously required a human touch, and it is precisely where an AI-assisted reviewer can be useful.

In the past, I was part of a group of engineers responsible for doing random code reviews with an eye for design system adherence. The design system team had been tracking who used their system the most and invited those engineers to help them review PRs, with the hope that education would be the key to getting more product engineers to use the system.

I wanted to create something like that here at Honeycomb for our design system, Lattice. My teammates encouraged me to consider an AI-assisted code review as part of that, and that's how Lattice Watch started. The impact it's had on our design system adherence has been incredible! The best part is that it's totally replicable to any kind of system you're implementing. You can use it for security, privacy, accessibility, and performance. Here's how we did it and how you can too.

Setting up the AI code review

The AI review is probably the part that has the most coverage online right now. I had previously set up a Dependabot review process that I wrote about here, and I followed a similar process of adding a workflow that gets triggered on every commit to our frontend repo:

• It goes through the diff and filters for the right type of files and locations to save some time and money.
• It gets reviewed by Claude with a structured prompt that covers the violations we're interested in catching.
• It then posts a comment on the PR explaining the issues and suggesting the most probable fix.

This complements our linting because it catches the code with context, allowing us to track complex issues. We made the reviews non-blocking, but any karma we saved, we spent right away making the reviews run on every commit of an open PR. This is annoying, but the consistency helps us track usage and see if PRs get better over time.

Using Honeycomb to understand Lattice Watch patterns

Alone, this process is helpful because product engineers get actionable feedback on their PRs. But it gets a lot more powerful once you start aggregating what it finds. At the end of each review workflow, we send telemetry to a Honeycomb dataset logging details of the review, like PR number, author, violation types, and files touched. The design enablement team uses this information to keep track of patterns and surface trends. We’ve now added Lattice Watch as one of our Goalie responsibilities each week.

I added a Board in Honeycomb that tracks useful queries, like the rate of violations to see if PRs are being opened with more violations than before (maybe something else in the harness broke? Maybe something happened with the prompt?), or which open PRs have the most violations (so our goalie can pop into a PR and give personal help to the author if they need it). When you start tracking this kind of information in aggregate, you're getting useful feedback that can improve your system and build an engineering culture where people find the system easy to use and friendly when they make mistakes.

Using Canvas to surface insights

The catch with telemetry is that someone still has to sit with the data and pull the insights out of it, and that takes time and experience. Honeycomb Canvas makes that part fast for me. I ask natural-language questions against live data and get answers back, like:

• "What interesting insights have you learned from the last 24 hours?"
• "Are violations going down over the life of PRs?" (i.e., are people fixing before merge?)
• "Who could I help based on sustained violations?" (i.e., identify teams to pair with)

By asking these kinds of questions, we've been able to see what common deviations we can patch up, as well as errors in Lattice that we can fix. For example, once the reviews started coming in, we quickly noticed that a common deviation was people rolling their own flexbox components. We updated our Lattice Watch prompt to call that out as a violation and suggest using Lattice’s Flex component. Now, we have more PRs using Flex, and any changes we make to that component flow downstream.

We’ve also used the Canvas insights to identify components with legacy APIs that didn't follow our new patterns and that tripped up agents trying to follow the new pattern. We were able to identify which components we need to improve, and agents can follow API patterns more easily now. Another time, while I was doing a manual review of a PR with a lot of violations, I was able to spot that we had one of our linters set to "warn" instead of "error," so we were allowing a lot of drift in tokens. I was able to patch this up temporarily by adding language to our Lattice review prompt, but our team also prioritized fixing that error in our linter.

By using deploy markers whenever we change our prompt, we give context to Honeycomb Canvas so it knows when we launched the process officially and when the trends and numbers change because we tweaked the prompt. It feels like I have a whole data analyst team working with me to improve the process, and I love the flexibility of changing the process as trends change (like more agents coding).

Guardrails that keep up

As more PRs are opened, either fully generated by agents or assisted by them, the replicability of this process becomes the glue keeping systems and prod aligned. You can use this pattern to maintain the guardrails of any system: first a linter to catch the easy, low-level fails, then an AI code review for the judgment calls, and finally telemetry to keep an eye on the trends of the system. We have this in place for our design system, but you could apply it anywhere you're trying to hold a line, like security, accessibility, performance, or whatever your team cares about.

Linters have caught obvious, repeatable drift for years. What's new is being able to run reviews that actually reason about context, and being able to look at your own data in Honeycomb to check whether those reviews are catching the right things.