Our iOS App Crash Rate

Our flagship iOS app crashes occasionally. We use Crashlytics to track crashes and Google Analytics to track usage which is a fairly common combination. I look at the crash reports every day just to see if anything unusual happens and maybe find something I can fix.

Although I never identify my employer (directly) or the products I work on, I thought a little analysis might be interesting instead of the usual essay.

Our apps are free and tend to be used at most once or twice a week to buy stuff. Sometimes people only use them once or twice a year and the average session for the iPhone app is only a few minutes. Doesn't sound like much but we make decent money per session on average.

This iPhone app analysis is for an iOS 7 only app and is the most recent release.

In the 7 day period I am looking at there were 60,000 sessions from nearly 40,000 users. Over that time period there were 56 crashes (21 different ones) which affected 55 people. So a crash rate per session of 0.09% and per user 0.14% or 1 in 714 users.

1/3 of the crashes are a particular one in MapKit where Apple's tile loading code in VectorKit appears to try to update a map which has already been deallocated. I reported it in Radar but so far no idea if they are going to fix it. It appears that when a map tile load isn't delivered fast enough it can complete after the app has already removed the map view.

Another 1/3 appear to be issues in iOS in low level networking code which are unidentifiable as the stack trace is purely in iOS and seem to be involved in cleaning up a socket connection. It could be something I am doing but there isn't any evidence. It's possible that the issue is network dependent but there isn't any easy way to know.

Of the remaining 1/3 which are rooted in our code, there are a couple of UI related bugs which I might take a look at fixing. The remaining bugs are all single bugs which are related to the oddball nature of travel data. Despite all the protective code we write we have to deal with the myriads of backend servers and all of the data from hundreds of suppliers finding some way to screw up in an original way. We try to fix these when they appear but many of these never happen again.

One of my favorite data errors was when the latitude would be randomly in the 1000s even though the longitude was correct. MapKit would toss a fit so since I can't know what the right value is, I just use 42 which puts the pin at the wrong location but at least it's not crashing.

Before shipping each version I always run the app through the Analyzer which is fairly decent at finding stupid code problems. The biggest change in quality was when we started to use ARC in new apps starting 20 months ago. With ARC it's much harder to make memory errors which are the most common cause for crashes in iOS apps in the past. We also use Veracode for security and PCI testing; this does a static binary scan looking for known security vulnerabilities but can also occasionally find wonky code.

Our old iPhone app spent a few years collecting a ton of crashes every day. Since I've been here the past two years we've taken a more active role in reducing problems in all of our apps. Our older apps of course used manual memory management which were the biggest cause of crashes. Use of the Analyzer made a big difference when XCode 5.0 shipped as it became pretty clever in finding and documenting those memory errors and helping us fix them. I believe it uses much of the ARC machinery to understand how you use/misuse manual memory management code. But replacing the older app with a brand new app reduced the crash rate from fairly unacceptable to almost imperceptible.

I hope this was useful to someone. There is no such as thing as crashless software especially given that you have to run it on an OS you have no control over. But reducing the problem to a bare minimum is always a good thing. In any case tracking and analyzing any problems is a must to keep customers happy.

I'm happier too when crashes don't happen!