Marketplaces Matter and I’ve got the analytics to prove it

Background

As I've covered many times in earlier posts, I've used PreEmptive Analytics to instrument a family of mobile yoga apps from TheMobileYogi. These apps are deployed across iOS, Android and Windows. The yoga apps are packaged in a variety of ways. Two apps – Yoga-pedia (free) and A Pose for That (premium) – are direct-to-consumer using a “freemium” model that includes embedded ads inside yoga-pedia. There are also a white-labeled app platform that can quickly generate a “re-skinned” app personalized for yoga studios, retailers and other “wellness-centered” businesses. And with all of these combined, I’m happy to report that we've passed the 110K download mark and still growing by the thousands each week.

The Issue at Hand

One adoption/monetization “variable” that is rarely measured in a clean way is the impact/influence that an app’s marketplace can have on the success of the app itself. This is in large part a practical issue – it’s not easy to compare, for example, Apple’s App Store with Google Play because the apps themselves are often quite distinct from one another – and so isolating the marketplace influence from the apps themselves can be tricky. However, with Android, we publish identical apps through two very different marketplaces; Amazon’s Android App Store and Google’s Google Play marketplace. By focusing on apps that are identical in every way BUT the API calls to the respective marketplaces, we can start to drill into the direct and indirect consequences of marketplace selection. 

Android makes up roughly 51% of TheMobileYogi downloads.

Android downloads combine both Amazon and Google Play adoption.

Android Downloads of Yoga-pedia

As of January 29, 2012, the total downloads of Yoga-pedia were:

• 21,109 Amazon (36% of the total) 
• 36,981 Google Play (64% of the total) or said another way, 

Google Play downloads were 75% greater than from Amazon.

…But downloads only tell a very small part of the story. What are users doing AFTER they download the app? How often do they use the app, for how long, and what exactly are they doing when they are inside?

Yoga-pedia Sessions

Using PreEmptive Analytics Runtime Intelligence, we see that there are in fact striking differences between the Google Play user population and the Amazon user population.

One glaring difference is the total number of users in each community.

The total unique users of from Google Play is 208% higher than that of Amazon.

If we were to stop here, I think our conclusion would be obvious – Google play delivers more downloads and more unique users than Amazon – and that has to make it a clear winner right? (Note, there has been no difference in marketing, advertising, etc. between the two marketplaces – specifically, we have done none).

…but if we were to stop here, we would be making a very big mistake!

How much time is spent inside the app? 

Another glaring difference that our analytics reveal is the difference between the average session length of our users – Amazon users tend to stay inside the app almost 3 times longer!

So – if we multiply the total number of sessions by the average session length, we can calculate how many hours were spent inside Yoga-pedia.

• Amazon: (41,937 sessions) X (13.88 minutes per session) = 9,701 hours 
• Google Play: (75,346 sessions) X (5.5 minutes per session) = 6,907 hours 

Total time spent inside the app distributed through the Amazon marketplace is 40% higher than from Google Play. 

If I am trying to maximize ad impressions, establish a brand or hold my user’s attention toward some other objective, Amazon now looks significantly more attractive to me than Google Play.

User behavior

Since Amazon users spend so much more time inside Yoga-pedia – how is their behavior different and how does that translate into measurable value? 

Returning users

Returning users (in red) form the majority of the Amazon session activity – Google Play users are less likely to use the app multiple times – they are “tire kickers’ for the most part. Returning users are roughly equivalent across the two marketplaces even though there are many more Google Play users overall.

Returning users are loyal and a lasting “relationship” can be established – whether you’re selling something, hoping to influence their behavior, or tap their expertise – recurring users are always “premium.” 

Ad Click Through Rate (CTR)

Moving to a more concrete metric – we can compare total impressions, Ad Click through Rates (CTR) as well as Ad Server Errors – for this analysis, we’re just looking at 30 days. Note: in both cases, the apps use AdMob.

	Google Play	Amazon
Ad Impressions	53,462	36,625
Ad Delivery Failure	1,853	425
Ad Failure Rate	3.47%	1.16%
Click Through Count	325	603
CTR	0.63%	1.67%

Amazon CTR is 164% higher than the Google Play CTR 

Google Play Ad Delivery Failure rate is (ADFR) 199% higher than the Amazon ADFR 

Now, it’s not really possible to isolate WHY these differences exist – but we can make some educated guesses. For CTR percentages – are Amazon users simply more conditioned or likely to buy stuff as compared to the typical Google Play user?

For ADFR percentages, we’re using the same ad service API, so the ad service itself is not to blame. Are the devices being used by Google Play users (as a total population) of lower quality or are they connecting through networks that are not as reliable?

Regardless, that kind of conversion delta is nothing to ignore.

Upgrades 

As I've already mentioned, in addition to pushing ads, Yoga-pedia is one half of a freemium model where we hope to get these users to upgrade to our commercial version, A Pose for That.

With PreEmptive Analytics, I’ve instrumented the app to track the feature that takes a user back to their respective marketplace (positioned on the app upgrade page). The ratio of unique users (not sessions) to upgrade clicks tells another important story; how likely is an Amazon user versus a Google Play user to upgrade to our paid app?

	Google Play	Amazon
Upgrade Marketplace	3,253	1,620
Unique Users	35,312	11,447
Conversion Rate	9.21%	14.15%

Amazon user conversion rate is 54% higher than the Google Play conversion rate. 

User behavior within my app

Yoga-pedia offers its users two locations where a user can click to upgrade; in a “tell me more” about the premium app page and at the end of an “Intro” to the current Yoga-pedia app.

By looking at the split of where users are more likely to “convert,” we can learn something important about the app’s design in general AND the differences between user patterns across marketplaces in particular. As a proportion, Amazon users are more likely to convert from the Intro page than their Google Play counterparts. The Intro page is “deeper” in the app (harder to find) and so this difference in usage pattern may imply a more thorough reading of embedded pages by Amazon users (and this would be supported by the much longer session times).

Exceptions 

Exceptions not only interrupt a user’s experience (with all of the bad things that flow from that), they are also a material expense (support, development, etc.). Given that we are talking about two virtually identical apps – would we expect one version to be more unstable (and therefore expensive) than the other?

	Amazon	Google Play
Sessions	41,937	75,346
Errors	1,523	3,150
Errors per session	3.63%	4.18%

Whether or not we expected it, the Google Play version of Yoga-pedia has an error rate per session that is 15% higher than its Amazon equivalent. 

Again – the analytics at this level can’t tell us why – but we can still make an educated guess regarding the differences in phone type and network stability of the two populations. 

Detail 

Of course, if you want to drill down into the specific exceptions (and examine stack traces, device types, carriers, etc – all of that is available through analytics as well.

Here are exception details for the error rates described above. Anyone want to help me debug these?

Do marketplaces matter? Of course they do. 

Of course, different apps will yield different results – but I don’t think that there can be any question that each marketplace comes with its own unique bundle of user experience, service level, and general appeal – and that, taken together, these attract their own distinct constituencies (communities) with their own behaviors, likes, dislikes and demographics.

App developers who chose to ignore the market, commerce and security characteristics that come with each marketplace will do so at their peril – the differences are real, they should influence your design and marketing requirements, and they will undoubtedly impact your bottom line and your chances of delivering a truly successful app.