Analytics and Mobile KPIs for data-driven decision making

Mobile App Analytics


In this Episode of the App Business Podcast – Chris & David discuss:

In the 2nd of our 3 part introduction to mobile app analytics, Chris and David discuss what metrics to track for maximum insight into performance of your app – mobile KPIs or Key Performance Indicators.  Since the goal of both the podcast and the series is to uncover actions that provides significant returns of both time and financial investments – a focus on how to get started and what to do with collected metrics are covered including:

  • In the News covering David’s new favorite old book and Apptopia’s new “code templates” offering
  • The various types of metrics to collect,
  • How to take immediate action for collecting this data
  • ….and what to do with this data at a high level

Resources and Linked Mentioned in this Episode:


Transcription:

Chris:  Welcome to the show, guys. Today we are going to be talking, it’s kind of our second episode on analytics, our second of three. We’re going to be doing a short three part series, kind of a quick overview on analytics and mobile KPIs. Ultimately the idea for these series is how do we make better decisions on what to do with our apps, whether it’s what apps to make, or how to improve our apps, or what to do next with our apps, which apps to re-invest in, data-driven decision making with mobile apps. It’s really important, but you just can’t jump into that. You need to know what you’re working with, what tools, and so, David, as you remember, last week we talked about the different services you can use, and some of the custom events you could create.

This week, we are going to take about Mobile KPI’s, which if you remember from last week, key performance indicators, and all the different things you can track, and the three general types of Mobile KPIs or pieces of data and some of the different services you can use and how you can implement these things. So we’re trying to give you the whole, from the beginning to end how to kind of implement the stuff that we talk about each week, right?

 

David: Absolutely, man. And just to underscore your point at the beginning, the underlying theme of maybe every episode we did so far is really we want to get to the next action. What’s the next action? That’s the fundamental question, and the whole purpose of getting more information in this case through analytics or through tracking mobile KPIs is just to inform a decision that you make what the next action is going to be. So I think that’s a really important thing to realize unless you have no decision to make or there is no next action defined, there is no use in tracking anything. That’s just the underlying concept I guess.

 

Chris:  Yeah, and this type of building products or services is pervasive across all sorts of startups right now, whether it’s using data to help guide a lean startup in the development cycles to the marketing with AB testing. There is just so much going on with data-driven decisions. Don’t trust yourself. Don’t trust your opinion, understand the data.

 

But for us in apps, we have to actually set up the means to track this data. So it’s not like a website where oftentimes you could just let Google tell you the important stuff. Right now with mobile, there is an awful lot that you have to set up, because so much is specific to your app, let’s say.

 

Okay, but before we jump in, we started a news segment, or what’s in the news, or what’s happening, what’s interesting. David, besides for that awesome book that you’re reading, which isn’t anything to do with mobile, but I guess it does have to get with getting stuff done. In fact, is it called Getting Stuff Done?

 

David: It’s called Getting Things Done by David Allen. A lot of the listeners would probably laugh because it’s such a classic, and so many people know about it, but I have made a New Year’s resolution to read and implement the book, and it’s quite an adventure. I like it. So if any of the listeners have problems with getting things done or want to boost your productivity, I really recommend that book. So, again, it’s Getting Things Done by David Allen. It’s going to be in your show notes.

 

Chris:  Yeah, so we were connecting after the holidays and saying, “Okay, what are our plans for this and that.” And I’m telling you, 50% of the conversation was, “Yeah, but so this book and have you read this book. Yeah, it’s really this book, and this book says to do this.” So it’s obviously informing and impacting the New Year for David, which is awesome.

 

David:            Yeah. Absolutely, but besides that, we did hear a bit of news from Apptopia which is a website and a company, I guess, that focuses on apps. Usually what they did is they sold and, so it’s a platform for selling and buying apps. It’s kind of a trading apps site, right?

 

Chris:  Yeah, a market place.

 

David: Yeah, but now they just added a new functionality or a new thing to their portfolio which is templates, I guess, is what they are calling it. And that’s just source code and documentation for apps basically that you can buy and then reskin and sell as your own apps. So that’s the big news there, but Chris, seriously what do you think about that? Isn’t that going to be just a rip, and rip again, and rip again cycle where everybody is going to make the same apps?

 

Chris:  Yeah. It certainly could be that way. I’m on a Mastermind call. You’re on the same Mastermind group, and we were talking yesterday about is that just going to lead to even more just junk in the app store and a ton of competition because it lowers the barrier to entry so far. And, couple that with Apple’s seemingly new, well, they’ve had this approach for a while but they seem to be tripling down on it, which is the rejection of no spamming the app store, where I’m getting rejected for my own apps. This isn’t code I bought, I created this app. Like a recent example is I created a pro version of free apps and it got rejected. So they are really big on cutting down on this.

 

Meanwhile, it seems like app companies like Apptopia are doubling down on selling pre-scrubbed code. I think it’s all going to end up in, it could be a black eye for some of these companies where a newbie comes along, buys this code which is probably a good place to start, submits it, it gets rejected. “Hey Apptopia is ripping me off. They’re scamming me because this code gets rejected every time.” I think it lends itself to that.

 

David: Yeah, for me, it awfully looks like a Gold Rush phenomenon where everybody seems to have heard something about reskinning and how easy it is and how you don’t need to know any code, etc., etc., and they seem to sell it especially also on their website right now. They sell it as a sure thing, so you just buy this code and you will be making profit in no time, and it really feels like Gold Rush. They are selling the shovels and the pick axes, and I think that the issue here is that reskinning really worked and may be still working, and it worked for some time and for some people really well. But as soon as something gets this mainstream, where you can conveniently go to a site and just buy the code, and it’s all nicely documented, and laid out for you, then I think, as you said, the competition is already so high, because everybody else can do it, too, that there simply isn’t a return anymore on your investment. Because what are you expecting if you pay $200 for some source code? Every monkey can do that and paste it to the app store. Why should your app bring any revenue?

 

Chris:  And it really comes down to the equation really of life is if everybody can do it, are you changing anything about it? Are you adding any value to it? Let’s say you’re adding a bunch of new monetization techniques or you have a really cool marketing technique, but if you’re just coming up with a cool idea and painting some new graphics on it, and not changing anything else around, then the competition for that exact same game play is going to be very high, and you’re just not adding a whole lot of value, and I think most people would recognize, “Yeah, I just need one to learn and then I’ll add value.” But the world doesn’t conspire to send you money, so you have to figure out a way to, if you’re adding value and something is different, you’ll get rewarded for that value.

 

But if you submit 10 apps, and you’re trying to play the numbers game, and none of the 10 are unique enough or different enough, then you could just strike out with all ten, and there goes $2,000 or $3,000 and now what? You didn’t learn anything. You might have learned a little bit, but you didn’t take any chances. I really believe in reskinning for a complete beginner, but they should still not just completely take the easy way out. They shouldn’t just buy the pre-packaged designs and use the same code. They should go through the experience of hiring a developer, hiring a designer, and working with those people, because those are the things that, and learning the monetization technique and all of these different things where you can improve apps on your own terms and not just taking some cookie cutter and playing against the 400 other versions of that apps that are out there.

 

David: So I guess be advised that you could lose your money when you buy these templates. You could potentially also add a lot of value and still make a lot of money with them. But you have to kind of do something different than all of the others in your market in the competition.

 

Chris:  And if you want to use a lot of this stuff that’s in a template for your app, I think that’s a great idea. But just copying the gameplay and copying every single element of it, and just slapping on some new graphics and calling it “Candy Crush Runner” is not going to be enough. Most likely, it’s not going to be enough. Maybe it was for these guys, but they were the first ones that submitted the codes. So the numbers you see is the first one to market. Now imagine if you’re the 50th one to market, what are your numbers going to look like? That’s the math that you have to do and really hold yourself accountable to are you adding any additional value and if not, then what do you expect to be rewarded for?

 

David: So once you have your app out there be it via a template, reskin, or developed yourself, then you’re probably going to ask yourself, “What do I need to track when we talk about analytics, which mobile KPIs do I need to track, and what do all of these KPIs actually mean?”

 

Chris:  What a segue. Love it.

 

David: And for the purpose of this episode, we have defined three general types of mobile KPIs or metrics. So that’s the macro type, the user type, and the monetization type, and we’re going to just go into the details of what metrics belong to these general types and how they relate to each other, I guess. Yeah, let’s start with the macro type and this can also just be called the general apps performance or how your app generally does in the store.

 

Chis:   Yeah and you know, I’m not aware of any industry terms, so we’re just going to throw these out there, macro or app performance. I mean, monetization is clear. User is kind of ambiguous, too, because most people when they see the user analytics, that’s what analytics are. So there are like three or four different types of metrics. They kind of overlap, but based on how you collect them, they kind of fall into three distinct groups for us. So I think that’s kind of a good way to talk about it.

 

David: Right, and the most, I guess, the KPI everybody knows about is the number of downloads that your app has and it falls under a macro or app performance that’s pretty easily trackable. Like for example, iTunes Connect will show you your download numbers, but it can also connect your iTunes Connect account to something like Appannie.com, and they will have a prettier interface that also displays your downloads, but it’s really the most general number that tells you how your app performs. That being said, it can also be very deceiving because a lot of downloads doesn’t mean a lot of revenue, right?

 

Chris:  Right. So I guess maybe we just rattle off the top group here and let’s call it the App Performance or Macro. It’s the App Annie kind of data, iTunes Connect type data or AppFigures is another one that I don’t use, but I think a lot of people use. But I really like App Annie, and they are always improving.

 

But you simply go to App Annie, log on with your iTunes Connect data or you connect that account, or in Google Play or Amazon, and it will show you downloads, revenue, updates. It will even show you your ranking in various stores, meaning the US and other markets. It will tell you, “Hey, you ranked in the top 100 in 89 global stores,” so it’s kind of fun to see that stuff, and then it tells you ratings and reviews.

 

So a lot of that stuff doesn’t tell you what is happening inside of your app, but it tells you how your app is performing at a macro level. You’ve got downloads, and here are how many people are updating it, and here is your revenue of in-app purchases, so all of your sales data, and it actually started to implement ad network data as well. So you can see some of your ad network data and all of your sales and in-app purchases. So your paid apps and your in-app purchases sales.

 

David: So if you do connect your app network to App Annie, you can actually see the total revenue as well and also split into three categories in the case of App Annie. That’s the sales revenue, the in-app purchase revenue, and the app monetization revenue. Overall, it’s just the whole total revenue or store revenue and that’s also a very important key metric of course, because if you have a lot of revenue you probably don’t care that much about the downloads, but the other way around, it wouldn’t be very nice if you have a lot of downloads but don’t get any money.

 

Chris:  Right. Okay, so that one is pretty clear. I’m guessing a lot of people, that’s the first thing they do as far as analytics and maybe their only thing. There is a stat only 50% of apps have any other analytics like an analytics package included. No one is tracking user data, which is insane to me.

 

So this next bit that we’re calling user data, what’s happening inside of your app, and what your users look like, their personas, and what their devices are and stuff. This is what most people think of when they think of app analytics. These are companies like Google Analytics and Flurry. There is a ton of them, Apsalar, and we’ll add a bunch of them to our show notes, and add a couple of other interesting links. But this is where a lot of the meat and potatoes and helpful information for decision-making comes in. Dave, do you want to rattle some off?

 

David: Sure. So OS version and device is something that is automatically collected by all of these analytics programs or libraries that you can use, and it tells you something about which device versions you need to support in the future and which OS version is dying off and things like that. It is also very important the number of active users and [inaudible 14:22] messages by different timeframes. So it could be the daily active users or the monthly active users, etc. Then there is the number of sessions and of course there are different ways to define what a session is but usually it’s just every time someone opens your apps so they tap on the icon on their home screen and the app opens up, that is counted as a session. Then you can subdivide this into sessions per user, the time a user spends every session on average, these kinds of things that can all be tracked easily by all of these analytics frameworks.

 

Then you can also look at the ratio between visitors or users that are new and the ones that are returning. Of course, this tells you something about how many people actually keep going back to your app versus the ones that opens for the first time. Related to this is also the loyalty value, which basically tells you how many people opened the app a certain number of times or the number of sessions after the nth occurrence, so this is something that also I guess automatically is tracked by most framework. Then you can have on top of that crash reports, load times, other technical info and of course all of the custom stuff that you can still configure in Google Analytics that would be called “custom events” or you can set goals or conversion goals, and all of these kinds of things.

 

Chris:  Yeah, and real quick, just to go back to the new versus the returning loyalty. So every app has its own kind of path, but new versus returning and loyalty are kind of a poor man’s engagement and retention report, so it can tell you how many people are engaged. New versus return is how many people you are retaining in a very loose way and loyalty is how engaged are they.

 

But really to answer that question, completely, you need to add custom events, and that’s tying something. I always like to think in terms of game levels, whether it would be a quiz, or a first person shooter, or a runner or something. How far are they getting in your app, and you can add different events to the different levels? So you can say, “Okay, of a 100 of downloads, 95% open the app,” and that’s a default stat. You don’t need to code that in. But then you’ll say, “Okay, 93% of those 95 completed level one, then 87% of those 93 completed level two,” and you can keep going through and seeing where you’re losing people, and where you might need to modify your gameplay, throw in some free rewards, add a tutorial, add some explanation.

 

There are all sort of tools in your toolkit that you can use to affect these numbers, but you don’t even know what to do without some of this data in front of you. So this is the most complicated by far of all of the analytics, which there’s the default data, which is really good, but the custom event data is really, you need to think about what a user would be doing in your app. Most of the time it’s not that complicated if you just map out what the app is and say, “Okay, did they get to level two?” These basic questions can be answered pretty easily by adding custom events, and it’s like shining a flashlight on a user in your app and really telling what is the general user experience for people downloading my app. Lots of time 100 downloads a day, that’s a lot of significant data that you can get from implementing this stuff.

 

David: Yes, custom events are absolutely powerful. I do know that Google Analytics has something called User Flow, I guess, where they actually can display the flow of your users where they are leaking out, at which screens they are leaking out. They are trying to detect what is a screen. So for really simple apps I think that’s also possible to use as well, to see where the users leak and stop using the app.

 

That’s not just for games, of course, because you’re always talking about levels, but it can also apply to “regular apps” where for example, you have a tutorial at the beginning of the app, and you can see with your custom events that only a certain percentage actually makes it through the tutorial. So this will tell you something about your tutorial, whether you should keep it, etc. So this is very important for any app that needs optimization.

 

Chris:  Yeah, I can’t wait to use Google. I’ve only used Flurry at this point. Well, I’ve tried Apsalar as well, but Google looks like, in the last several months, they’ve made some pretty significant improvements. I’m looking forward to putting them in my next app and seeing what they’ve got. I’d love to get comfortable using Google, because I know they’re not going to go anywhere. Flurry seems like they are really actively selling ad data and that’s where they make money. But I think Google, obviously, is in it for a very, very, very longterm. We’ll see what they do.

 

Okay, so then out of this, once you have all of this data, you can start segmenting your users based on various metrics. How many made it to level two? How many have played the game five times? How many have done X, Y, Z? So you can start pulling together groupings based on all of this stuff that you are collecting. You can also, based on your spend, let’s say you’re doing a Facebook ad or you’re doing a cross-promotion, you can see, you’re just posting it on, you’re sending an email anywhere, you can create a custom link for that acquisition target and then see of those people coming from that source, how many are getting X far. Does an ad on Facebook, does that user look the same as a user acquired through the app store?

 

So you can start to see if your marketing spend is valuable and all of that stuff. That’s probably when you start making $0.50, $1 per download, and that’s a great number, and that’s a little ways away unless you have a hit app. So without digging too much into that, you want to just jump into monetization, David?

 

David: So monetization. Yes, that’s basically everything that has to do with money. That would include in-app purchases, and I think very important to just distinguish here is also the general and the purchase revenue and the general stuff, that’s all fine, but in the monetization category, we are also looking at which items specifically generate revenue. So you could have a number of in-app purchases in your app, and some in-app purchases are really inexpensive; others are very expensive, so on and so forth. Then you could look at the individual items and see, okay, the expensive one is actually not really generating a lot of revenue, but the cheap one, or the other way around, etc., so you can, again, optimize, optimize your pricing strategy, etc. This is something that will really inform your decisions if you’re using in-app purchases at all.

 

Then there is of course your ad network’s fill rate. This would also be provided by your ad network in most of the cases. And there is the metric called ECPM, which we just briefly discussed before this episode, which should actually be called revenue per 1,000 requests because ECPM is a term that we carried on from the advertising world in terms of something like Google AdWords, and it tells you the estimated cost per 1,000 clicks inside something like a Google AdWords campaign. But that’s, of course, from the side of the advertiser, who wants to buy ads and not from the side of the seller who is selling advertising space, and that would be our position as an app property owner. So there is always a difference between the ECPM that the advertiser pays and the revenue that we get, because that’s exactly the kind of the money that a company like Google or any other ad network makes. That’s their revenue.

 

Chris:  Right. I was going to say RPM or revenue per impression or 1,000 impressions, Google, I think, is starting to use that term with the mobile analytics, and I’m glad to see that because ECPM is really a weird term for us to think in. It’s not estimated. It’s actual and things like that. So, yeah, the RPM number is a better number for us to think in, but if your ad tools are using ECPM, and most of them are, it’s the same thing. Well, it’s not the same thing, as David just explained, but that’s how they are using that number. ECPM is how much you’re earning per 1,000 impressions. And again, it’s not the accurate way of describing that, but I think the industry is starting to move because Google is moving, to referring to it as RPM, revenue per 1,000 impressions, which is the number we care about.

 

David: Right. So don’t be confused if you see “ECPM”, it’s basically is the same number although it’s not labeled the right way.

 

Chris:  I think we might have made that more confusing. I’m not sure.

 

David: Yeah, so then there is one more really important metric and also not a very important metric, depending on what your situation is, and I’m talking about the lifetime value of your users. So each user has a certain lifetime value. If you average that out that’s your lifetime average per user. I think most people have heard the term “lifetime value”. Basically it tells you what the revenue per user is over a long period of time, however you define that. What really a lifetime is isn’t really defined, and what it means is the lifetime of the user.

 

Chris:  Yeah, if they are around for nine months and they might buy three times, maybe they are worth 3 times $0.70, so they’re worth $2.10, and so, okay, cool. The average user stays nine months and buys three times for $2.10. There is the lifetime value, which is great. It’s not an easy number to get in mobile, but if you have an idea of that number and can came come close to it, then it really can inform how much you can spend on user acquisition.

 

So the name of the game for scale, and this is how Clash of Clans and everyone else does it, is they make $4.00 per user or much more, but their lifetime value is let’s say $4.00, and they acquire users for $2.00. So who wouldn’t want to do that all day long? Of course if I spend $2.00 and make $4.00, here is $10 million. I’ll give you $10 million and you give me $20 million back. It’s a no-brainer, and that’s what LTV drives. That’s what lifetime value of the user drives is that decision-making where you can go and spend money, acquiring users, because you are going to make more than you’re spending.

 

David: Of course that sounds like the Holy Grail, just trading money for more money. But the catch is that you have to have the cash in order to buy the traffic and then it takes some time for that traffic to generate revenue, and in the case that you said, it was nine months. So you have to get the cash out of your own pocket for nine months before all of that will return. So if you’re just starting out, and if you don’t have a lot of cash, lifetime value will not be that important for you because you can’t actually make the use of it because you don’t have the money to invest.

 

Chris:  I’ll tell you what, for all of the listeners, if you have a lifetime value for over $1.50, call me, and we’ll figure it out. The real thing for most beginners is lifetime value is to have it to be a high enough number to justify acquiring users is not very likely for a new app designer or developer or new apps, and that’s what it really is.

 

In fact, let’s just jump to the next one, revenue per download. I kind of look at my business in a monetization rate, which is just purely total revenues divided by total downloads for an app. Let’s say I make $100 on 1,000 downloads, that would be 10%. And that’s kind of how I look at it. I look at, okay, I’ve got a 10% monetization rate. That’s with in-app purchases and ads, everything all in, how much revenue do I make divided by downloads.

 

Now, that’s not a very advanced way to track this, but it’s close enough. When it starts getting to 45%, 50%, 60%, 70% or $0.70 per download, then I want to really understand it better, because then I can start looking at, “Okay, how can I acquire users for less than that?” But until it gets above $0.50, $0.60, $0.70, you’re not going to be able to acquire any users at scale with those small bids of $0.40, $0.50, $0.60. I hope that made sense.

 

David: Yeah, what you just described, I think, is a really nice, really valuable tactic, and that’s also why I’m kind of criticizing the LTV metric. Because you can start to focus on collecting these metrics and really precisely and accurately calculating them, but if you don’t take any action on it, or if you can’t take any action on it, then it’s really just a waste of time, and you really want to avoid that. So rather than start with something really simple that you get really simply by just connecting your iTunes connector to App Annie or something like that, and make your decisions later. But you should at least know that these metrics are out there, that you can collect them. So that if you have something that’s interesting, in your case, something that makes $0.40, $0.50 per download, then you can say, “Okay, now I really want to hone in on this and really optimize it really well.”

Chris:  Yeah, and I think that’s a great way to close the episode. , the point of tracking all of this information is to inform our decisions, to have data-driven decision making, and a lot of this stuff is stuff you’ll grow into. I think what we were talking about before the episode started, David, was if you don’t have your play button working or if the text doesn’t wrap the way you want it to, and you’re not getting enough users for it to matter or you’re not getting any downloads, then how far they get into your app, doesn’t really matter. But as soon as you have the basic functionality working, you probably don’t want to invest too much in any next versions until you have a better understanding how a user’s interacting with your app.

And for the bang for your buck, which is what we are both very interested in ROI, the return on your investment of putting custom events in your app and understanding what users are doing to help inform your next versions and your development efforts, investments, or time is well worth the investment of putting the custom event in there and spending the time to understand how this stuff works.

David: Absolutely, let’s wrap it up for this episode. If you want to get the show notes for this episode, please visit our appbusinesspodcast.com/kpi. It’s all lower case, and I hope we can see you on the next episode where we will probably close this three-part series on our analytics. Thank you very much for listening.

 

Chris:  Thanks, guys.