RUM can leave questions unanswered.
Honeycomb for Frontend Observability doesn’t.
Mobile apps are a little different from services run on servers. You build your mobile app, you ship it off to the world, and then it gets run by the end user on their own machine. If your app is running poorly on some percentage of users’ devices, you may never know. That’s where observability comes in.
There are certain important metrics that every mobile app has in common. At Honeycomb, we have surveyed these metrics across iOS and Android, and have defined a set of Core Mobile Vitals we think every app developer should care about. The purpose of these vitals is similar to that of Core Web Vitals for web frontends.
At a high level, the Core Mobile Vitals include:
- App Startup Time
- Crash Rate
- Hang Rate
- Slow Render Rate
Our Honeycomb SDKs for iOS and Android leverage available signals to collect these vitals right out of the box. Our SDKs are built on top of the open-source OpenTelemetry SDKs. On Android, we take advantage of OpenTelemetry’s robust auto-instrumentation to measure these signals, in line with Android’s official recommendations. On iOS, much of the data is collected using MetricKit, which allows you to collect detailed daily snapshots of your app’s performance.
Let’s dig into what each of these signals means.
App Startup Time
Have you ever opened an app and waited so long for it to load that you forgot why you opened it in the first place? Not a great user experience. It’s also too easy to accidentally add code that unnecessarily increases startup time. For example, if you load too many assets synchronously in your app’s initialization code, it can cause the app to stall before the user has a chance to do anything.
An app’s startup time is generally measured from the time your app first has a chance to run your code up until the first screen has finished rendering. Sometimes it’s referred to as “time to first render.” There are different degrees of “startup,” based on how much of your app was already running.
- Cold: Your app was not running at all.
- Warm: Your app’s process was running, but the specific screen or Activity needed to be created.
- Hot: Everything was running, but needed to be brought to the foreground.
We consider startup times <400ms to be good, those in the 400ms-1.5s range to be medium, and those >1.5s to be bad. These numbers were derived based on discussions with developers and are also consistent with guidelines given by Google and Apple.
Crash Rate
The most disruptive experience a user can have is the app completely crashing. On both Android and iOS, our SDKs track unhandled exceptions and log them to Honeycomb. On iOS, we take that even further, by taking advantage of MetricKit’s crash diagnostics to capture other kinds of crashes, such as segmentation faults. With MetricKit’s app exit data, you can also break down all of your app’s exits by the reason for the exit and whether the app was in the foreground or the background.
Hang Rate
Sometimes an app doesn’t quite crash, but still becomes unresponsive for an unacceptable length of time. Typically, this is the result of doing too much synchronous work on the main thread, preventing your app from handling user input or redrawing the screen.
In Android, an unresponsive event is called an ANR (Application Not Responding). An ANR happens whenever an app becomes unresponsive for >5 seconds. OpenTelemetry provides auto-instrumentation that detects these events and automatically logs them, with a stack trace to show what the main thread was doing when the ANR was detected.
In iOS, MetricKit reports hang events, which similarly include a stack trace. MetricKit also provides a histogram of hang times. In Honeycomb, the histogram is reported as an average hang time, so that you can measure the overall responsiveness of your app. Apple has additional tips on how to debug hangs and improve app responsiveness.
Slow Renders
Even if your app doesn’t outright hang, if your app can’t render its UI quickly enough, it can feel janky.
In Android, any render that takes longer than 16ms is officially considered slow, whereas anything over 700ms is considered frozen. OpenTelemetry captures these events automatically. In iOS, MetricKit provides us with a measurement of hitch time ratio. A hitch time ratio <= 5ms/s is considered good, whereas 5-10ms/s is a warning, and anything >10ms/s is considered critical.
When digging into why rendering performance might be slow, we provide manual view instrumentation for SwiftUI and Jetpack Compose, to let you measure the construction and rendering time of individual UI elements.
Conclusion
While the exact definitions and metrics for these vitals may change over time, we believe the broad categories are—and will continue to be—meaningful for every app. By using Core Mobile Vitals to keep on top of your app’s overall health, you can prevent your users from getting frustrated with unresponsiveness. And Honeycomb’s mobile SDKs are committed to collecting this telemetry automatically, so that you can focus your manual instrumentation on the special things that make your app unique.
