In the seemingly never ending fight between iSheep and Fandroids, the main crux of the “debate” usually ends up around the operating system, with each side claiming to have the better one. Is one better than the other? We likely will never find out seeing as we haven’t even defined what constitutes a “better” operating system. Developed by highly-paid programmers, both are probably as good as the other — in their virgin, unadulterated forms. The problems people have with smartphones typically stem from third-party apps (or, in the case of some Android phones, a manufacturer who doesn’t know how to hire competent programmers… but that is a story for another time).

No one owns a smartphone for use in its bare-metals form, using only operating system-provided features. Apps are the most significant attractor to smartphones; apps are one of the biggest, if not the biggest, reason why iOS and Android are as successful as they are today. Indeed there are many mobile operating systems in existence (e.g. webOS, Bada OS, Symbian, Windows Phone, etc.), some of which are considered by some people to be “better” than iOS and Android; yet none of them are close to the dominance exerted by iOS and Android largely due to the huge third-party developer support both platforms enjoy. (For what it is worth, Windows Phone is slowly gaining market share.) So, then, a study looking at the stability of iOS apps vs Android apps would be handy, right? Well, hey, look at that — this article is  about a study done measuring the stability of iOS apps vs Androids apps. Funny how that works out.

The Study

Crittercism, a company that specializes in providing developers with analytics about their iOS and/or Android apps, recently conducted a study during November and December 2011 that measured app crashes. The study, which garnered data by studying 214 million app launches (162 million on iOS and 52 million on Android) during November and December, looked at app crashes on iOS and Android. The results may surprise you.

What constitutes a “crash”?

Before we get into the results, it is important to point out the study by Crittercism does not differentiate between an app crash where an app itself crashes and stops working and an app crash that crashes the device, forcing a reboot. In my experience, being an Android user, most app crashes that happen on Android are the former type — the app itself crashes, with a message being shown on-screen asking the user if they want to kill the app or not. However, there have also been times when an app crash has forced a reboot of my phone. So I’ve had both, albeit less of the device reboot app crashes. I’m not an iOS user – I don’t own any iDevices (feel free to buy me one) – so I can’t say if app crashes on iOS are more app-crash app crashes or app-crash-device-reboot app crashes. My guess is iOS is similar to Android in this regard — more of the former but still some of the latter.

The Results

+1 Android

The overall trend found by Crittercism is apps on iOS tend to crash more than Android. In the first quartile (the “best” apps on iOS and Android), Android apps had a crash rate of 0.15% while iOS apps did the deed 0.51% of the time. In the second quartile, the score was 0.73% and 1.47% for Android and iOS respectively. In the third quartile (the “worst” apps on iOS and Android), Android apps crashed 2.97% of the time while iOS apps came in at 3.66%.

The noticeable culprit across both platforms is iOS 5.0.1 which, in a normalized data set, constituted 28.64% of all crashes (“all crashes” is the total number of crashes combined from iOS and Android). Following iOS 5.0.1, iOS 4.2.10 had 12.64% of overall crashes, iOS 4.3.3 10.66%, iOS 4.3.5 8.98%, iOS 4.1 8.24%, and iOS 5.0 4.8%. The biggest loser for Android was Android 2.3.3 with 3.86% and Android 2.3.4 following close behind with 3.65%.

Google’s latest Android 4.1 is part of the “Android Other” category which had 0.54% while Apple’s latest iOS 5.1 had 0.02% of crashes. It should be noted, however, at the time of the study the latest version of Android released to the public was Android 4.0.3 (if I remember correctly) so the results for Android 4.1 may not be totally accurate; and the latest version of iOS 5 released to the public was iOS 5.0.1 so the results from iOS 5.1 may not be totally accurate. It should also be noted Google has released minor updates to Android 4 since the Crittercism’s study took place (I believe Android 4.0.4 is the latest public version at the time of this writing); Apple is still on iOS 5.0.1 at the time of this writing, with updates – such as iOS 5.1 – in Beta stages.

Android Analyzed

Of all the apps crashes on Android (note: the following numbers are for crashes on Android only, they are not the total crashes from iOS and Android), the most crashes took place on Android 2.3.3 (24.76%) followed by Android 2.3.4 (23.38%), Android 2.2.2 (8.34%), Android 4.0.1 (6.67%), Android 2.3.7 (6.65%), Android 2.2.1 (6.47%), Android 2.3.5 (6.19%), and Android 2.2 (4.95%).

iOS iResults

Out of all the app crashes on iOS (note: the following numbers are for crashes on iOS only, they are not the total crashes from iOS and Android), the most crashes took place on iOS 5.0.1 (33.93%) followed by iOS 4.2.10 (14.98%), iOS 4.3.3 (12.63%), iOS 4.3.5 (10.62%), iOS 4.1 (9.76%), and iOS 5 (5.68%).

While iOS 5.0.1 takes the crown when analyzing iOS app crashes by OS version, it is the iPhone that wins when looking at iOS app crashes by iDevice. Of all the app crashes that took place on iOS, a resounding 74.41% of the crashes took place on an iPhone, with the remaining crashes split about evenly between iTouch (14.87%) and iPad (10.72%).

The Discussion

I’m sure by now you must be wondering why Android apps seemingly crash less than iOS apps. So let’s discuss.

When it comes to comparing Android 4.x (the latest Android version at this time) vs iOS 5.x (the latest iOS version at this time), the key is adoption rate. You see new releases of each respective operating system brings new features and often under-the-hood changes that affect how apps operates and therefore how they should be developed. Developers need to update their apps to work with the latest version of each operating system, something that takes time. At the time that the Crittercism study was conducted, both Android 4.x and iOS 5.x were very new. In other words, many developers had not updated their apps to properly run on the latest version of each OS — a lot of apps that worked on Android 2.3.x and iOS 4.x crashed on Android 4.x and iOS 5.x. The reason why iOS 5.x has a much higher app crash rate than Android 4.x is simply because of a higher adoption rate; iOS 5 was made available to iPhone 3GS, iPhone 4, iPhone 4S, and iPad 2 owners whereas Android 4.x – at the time – was only available on Samsung Galaxy Nexus and Samsung Nexus S. To put it simply, a lot more people were running iOS 5.x than people running Android 4.x at the time of this study (and frankly, because of Android manufacturer’s poor track record in updating devices, many more people still are running iOS 5.x than Android 4.x). This is one of the reasons why Apple’s latest OS has a much, much higher app crash rate than Google’s latest OS. However, it isn’t the only reason.

If we look at the overall numbers, and for arguments sake nullify the whole new-OS-effect, apps on iOS still crash more often than Android apps. Why is this? Is it because Android is just a better OS? Or maybe Android has better developers? I suppose those could be possible reasons but I doubt it. There is no one simple answer to phenomenon; but I have three potential explanations:

• All apps have bugs, some of which cause crashes. It is up to the developers to fix those bugs and issue app updates. Being “open” (and I put open in quotes because Android’s openness is debatable), Android allows developers to quickly push app updates, pretty much in real time. Updates to iOS apps take much longer to appear — sometimes up to a week. Faster app updates = less app crashes (generally speaking).
• Bug fixes are only effective if people update their apps. iOS lacks the ability to automatically update apps; and, seeing as iPhone is a “common” phone in the sense that everyone has it and not just tech-savvy people that know to manually check for updates to their apps, this hurts iOS apps because the buggy apps are less frequently updated by users to newer, less buggy versions. Android, on the other hand, has the ability to automatically update apps and, in the cases when automatic update is not available, Android notifies users an app update is available, encouraging them to update their apps. The faster users update their apps, the less apps will crash (generally speaking).
• Android has a more competitive marketplace than iOS. I’m not saying Android users spend more money on apps than iOS apps. In fact, I am saying the exact opposite. Android has a more “free” culture when it comes to apps than iOS. Indeed stats show Android roughly has a 60/40 split on free/paid apps whereas iOS has the exact opposite. When it comes to free Android apps, it is advertising revenue that is at stake — that’s how developers make their money. (When someone pays for something they are more likely to stick to it because they feel like they invested something in.) Users that buy apps are more likely to use those paid apps versus if they got those apps for free. When it comes to buggy apps it is a lot easier to ditch free apps that are buggy than it is to ditch paid apps that are buggy because you already paid for the paid apps. By this logic, Android Marketplace is a lot more competitive than Apple App Store simply because developers need to create better quality apps to deter users from jumping to the next free app that provides the same function.

Other than that, I’m not sure why apps on Android crash less than apps on iOS. If you have any theories of your own, share in the comments below.

The Conclusion

Based on the study by Crittercism, Android apps crash less than iOS apps. So what does this mean to the end-user? Frankly, not much. True, iOS apps do crash more often than Android apps. However, the numbers – especially when you get to the first quartile or “best” apps – are so small that real-life users probably won’t notice any difference between either platform. The difference between iOS and Android, when it comes to app crashes, is fairly insignificant in my book. The bottom line is apps that crash will crash. Apps that don’t, won’t. This isn’t an OS-specific issue. Nothing to jump ship over.

Feel free to share your thoughts on this matter in the comments below.

[via Forbes]