March 01, 2015

Is MAU an effective audience metric?

instagram-user-growthThere was much hullabaloo in December when Instagram announced it had reached the milestone of 300 million monthly users, surpassing Twitter, and putting the latter under a bit of pressure in its earnings call a couple of weeks ago. But there has also been plenty of debate about whether these measures of the reach of major internet services are reliable, especially when comparing numbers from two different companies. Just what is a “monthly active user”, or MAU, anyway?

Defining MAU and DAU

Monthly Active Users is a pretty simple metric conceptually – it is the number of unique users who were “active” on a service within a given month. It doesn’t matter how many times each user used the service in the month; they’re only counted once (it’s a UU measure, after all). Daily Active Users is just the same measure, but over the period of a single day. So when Instagram says it had 300m active users in the Month of November, that means that 300m unique users did something in one of Instagram’s apps during the month.

Of course, for a signed-in service like Facebook, Twitter or Instagram, the total number of registered users will always be much higher than active users, since there will always be a significant subset of users who register for a service and then never use it (or have stopped using it). By some estimates, Twitter has almost 900 million registered users, almost four times the number of monthly active users. But registered users doesn’t tell you very much if you’re trying to run one of these services, at least not on its own – if it is massively out of whack with your active user counts, then it might indicate that your service isn’t very compelling or sticky.

Since journalists are also skeptical about registered user numbers, online services have taken to reporting MAU instead. These services have an incentive to report the biggest possible active user numbers, so tend to include almost any measurable interaction with their app or service in the definition of “active”. But from an analytical point of view, this isn’t the most helpful definition. Not every interaction with a website or app really represents “active” or “intentional” use. But how do you define “active” engagement with your app or service? That depends on what you’re trying to achieve with the metric. Let’s break it down.

MAU vs MEU

Let’s look at some of the things you can do with the Instagram app:

  • Launch the app
  • Browse your feed (just look at photos)
  • Look at someone’s profile
  • Follow someone
  • Favorite a photo
  • Comment on a photo
  • Post a photo
  • Post a video

I’ve tried to order this list from “least-engaged” behaviors at the top to “most-engaged” behaviors at the bottom. At one end of the spectrum, it’s almost impossible to use Instagram without browsing your feed (it’s the thing that comes up when you launch the app), so it’s hardly a reliable indication of true engagement (some fraction of that number will even be people who launched the app by mistake when they were stabbing at their phone trying to launch Candy Crush Saga from the icon next door). At the other end, users who are posting lots of photos and video are clearly much more engaged, and a count of these folks would be a reliable indication of the size of the engaged population.

So where to draw the line? That depends on what you consider to be the minimum bar for “engaged” behavior. At Microsoft we’re having some very interesting discussions internally on where and how to draw this line across our diverse range of products – “Active” use means something very different across Bing, Office and Skype, to name just three. The advice I am giving my colleagues is to set the bar fairly high (i.e. not count too many behaviors as active use). Why? Well, consider the diagram below:

image

The outermost circle in the diagram represents the entire population of users of a service. As we covered earlier, only a subset of these users could be considered “active” (i.e. actually use the service at all), and an even smaller subset “active and engaged” (use the service in a meaningful way). If you’re running the service, it’s this group of users, however, that you’re most interested in cultivating and growing – they’re the ones who become the “fans” that will promote your service to their friends, and (if your service has any sort of social or network quality) will actually contribute to the quality of the service itself (Instagram would be pretty dull if nobody posted any photos).

What this all adds up to is that if you’re looking to track the growth and engagement of your user base, you probably want to track a couple of metrics:

  • Monthly Active Users (MAU) [Active Unengaged + Active Engaged, above]
  • Monthly Engaged Users (MEU) [Active Engaged only]

Of these two, the really important one is the MEU – the a number that really represents worthwhile usage of your product or service, and which only includes behaviors that are the ones you really want to encourage amongst the user base. If I were working at Instagram, I’d probably include almost all of the actions in the list above (possibly excluding app launch) in my definition of Active Users; but I would only include “Post picture” and “Post video” in my definition of Engaged Users (I might be persuaded to include “Post Comment” since it does contribute to the network.

Tracking MEU has another couple of advantages: If the number goes down, you’ll know that engagement with your service is diminishing. You can also track MEU as a fraction of MAU: If MEU/MAU is only 50% you can focus on growing engagement in your active base, whereas if MEU/MAU is 95% (i.e. almost all active users are engaged), you’ll probably want to focus on growing the active base (by recruiting new users).

The tactics for moving MAU and MEU will differ. To grow MEU, you can market to your existing base of “active unengaged” users (the population who falls into MAU but not MEU). These are the lurkers or the casual users who may only need a little nudge to become truly engaged and move into the middle circle. To grow MAU, you’ll need to recruit new users to your service, either from the pool of inactive users, or from the general population. This is usually a harder nut to crack, and one of the best tools in any case is to use your base of engaged “fans” to recruit – which underlines the importance of growing the MEU number.

So a final benefit of using MEU is that it is likely easier to move than MAU; and the next time you’re standing in front of your VP going through your product dashboard, you’ll be glad you picked a KPI you can actually move.

del.icio.usdel.icio.us diggDigg RedditReddit StumbleUponStumbleUpon

November 09, 2011

Building the Perfect Display Ad Performance Dashboard, Part I – creating a measurement framework

dashboard-warning-lightsThere is no shortage of pontification available about how to measure your online marketing campaigns: how to integrate social media measurement, landing page optimization, ensuring your site has the right feng shui to deliver optimal conversions, etc. But there is very little writing about the other side of the coin: if you’re the one selling the advertising, on your site, or blog, or whatever, how do you understand and then maximize the revenue that your site earns?

As I’ve covered previously in my Online Advertising 101 series, publishers have a number of tools and techniques available to manage the price that their online ad inventory is sold for. But the use of those tools is guided by data and metrics. And it’s the generation and analysis of this data that is the focus of this series of posts.

In this series, I’ll unpack the key data components that you will need to pull together to create a dashboard that will give you meaningful, actionable information about how your site is generating money – or monetizing, to use the jargon.

We’ll start by taking a high-level look at a framework for analyzing a site’s (or network’s) monetization performance. In subsequent posts, we’ll drill into the topics that we touch on briefly here.

 

Getting the measure of the business

Ultimately, for any business, revenue (or strictly speaking, income or profit) is king. If you’re not generating revenue, you can’t pay the bills (despite what trendy start-ups will tell you). But anyone running a business needs a bit more detail to make decisions that will drive increased revenue.

In the ad-supported publishing business, these decisions fall into a couple of broad buckets:

  • How to create more (or more appealing) supply of sellable advertising inventory
  • How to monetize the supply more effectively – either by selling more of it, or selling it for a better price, or both

Another way of thinking about these decisions is in a supply/demand framework that is common to almost all businesses: If your product is selling like hot cakes and you can’t mint enough to meet demand, you have a supply problem, and you need to focus on creating more supply. If, on the other hand, you have a lot of unsold stock sitting around in warehouses (real or virtual), you have a demand problem, and you need to think about how to make your products more compelling, or your sales force more effective, or both.

Online publishers usually suffer from both problems at the same time: Part of their inventory supply will be in high demand, and the business will be supply-constrained (it is not easy to mint new ad impressions the way a widget manufacturer can stamp out new widgets). Other parts of the inventory, on the other hand, will be hard to shift, and the business will be demand-constrained – and unlike widgets, unsold ad inventory goes poof! when the clock strikes midnight.

So analysis of an online ad business needs to be based on the following key measures:

  • How much inventory was available to sell (the Supply)
  • How much inventory was actually sold (the Volume Sold)
  • How much the inventory was actually sold for (the Rate)

It’s ultimately these measures (and a few others that can be derived from them) that will tell you whether you’re succeeding or failing in your efforts to monetize your site. But like any reasonably complex business (and online advertising is, at the very least, unreasonably complex), it’s really how you segment the analysis that counts in terms of making decisions.

 

What did we sell, and how did we sell it?

Most businesses would be doing a pretty poor job of analysis if they couldn’t look at business performance broken out by the products they sell. A grocery chain that didn’t know if it was selling more grapes or grape-nuts would not last very long. Online advertising is no exception – in fact, quite the opposite. Because online ad inventory can be packaged so flexibly, it’s essential to answer the question “What did we sell?” in a variety of ways, such as:

  • What site areas (or sub-areas) were sold
  • What audience/targeting segments were sold
  • What day-parts were sold
  • What ad unit sizes were sold
  • What rich media types were sold

The online ad sales business also has the unusual property that the same supply can (and is) sold through multiple channels at different price points. So it is very important to segment the business based on how the supply was sold, such as:

  • Direct vs indirect (e.g. via a network or exchange)
  • Reserved vs remnant/discretionary

Depending on the kind of site or network you’re analyzing, different aspects of these what and how dimensions will be more important. For example, if you’re running a site with lots of high-quality editorial content, analyzing sales by content area/topic will be very important; on the other hand, if the site is a community site with lots of undifferentiated content but a loyal user base, audience segments will be more relevant.

 

Bringing it together – the framework

I don’t know about you, but since I am a visual person to start with, and have spent most of the last ten years looking at spreadsheets or data tables of one sort or another, when I think of combining the components that I’ve described above, I think of a table that looks a bit like the following:

image

This table is really just a visual way of remembering the differences between the measures that we’re interested in (volume, rate etc) and the dimensions that we want to break things out by (the “what” and “how” detail). If you don’t spend as much of your time talking to people about data cubes as I do, these terms may be a little unfamiliar to you, which is why I’m formally introducing them here. (As an aside, I have found that if you authoritatively bandy about terms like “dimensionality” when talking about data, you come across as very wise-sounding.)

In the next posts in this series, I shall dig into these measures and dimensions (and others) in more detail, to allow us to populate the framework above with real numbers. We’ll also be looking at how you can tune the scope of your analysis to ensure that

For now, here’s an example of the kinds of questions that you would be able to answer if you looked at premium vs non-premium ad units as the “what” dimension, and direct vs indirect as the “how” dimension:

image

 

As this series progresses, I’d love to know what you think of it, as well as topics that you would like me to focus on. So please make use of the comments box below.

del.icio.usdel.icio.us diggDigg RedditReddit StumbleUponStumbleUpon

About

About me

Disclaimer

Subscribe

Enter your email address:

Delivered by FeedBurner

Subscribe