Are UU numbers worth the bother?

Reading Time: 3 minutes

We’ve been having a bit of a discussion here of late about the cost/benefit ratio of providing ‘proper’ (that is, properly and accurately calculated) Unique User (UU, or sometimes called Visitor) numbers in web analytics reports.

Whilst UU numbers are useful and desirable (I don’t think anyone would argue that you can’t benefit from them at all), they come at a cost. And what’s more, the benefit they deliver can fail to be appreciated by users, even causing questions to be raised about a tool’s accuracy. So it is pertinent to ask whether it’s worth delivering UU numbers throughout your web analytics reports.

To expand, let’s take a closer look at the costs & challenges of providing UU numbers:

  1. Computational cost
    To calculate a UU count for a range of data, you have to count up the number of unique user identifiers that you find in the entire data set. This is a computationally expensive thing to do. If you’re designing a web analytics platform, you can do this kind of stuff up-front and cache the results, but if you want your tool to be able to offer UU counts over custom date ranges, you’ll always hit a point where a user asks for a UU count that hasn’t been pre-cached. This will be slow to deliver, and probably annoy users in the process.

    The reason for this is because UU count numbers are not additive over a date range. That is, if you know the UU numbers for each individual day of a given week, you can’t calculate the UU count for the entire week by just adding the day numbers together. This is because of people returning during the week on different days, who would be double-counted if you just added the days up. So you have to go back to the underlying data and recalculate from scratch, which is slower.

  2. Tool complexity/ungrateful users
    The real tragedy of UU numbers is that, after you go to the effort of calculating them, you then have to spend hours explaining to skeptical users why they’re important, and why the UU number for March is not simply the sum of the individual UU counts for all the days of March. I’ve lost count of the number of times I had to justify the numbers that WebAbacus was producing for unique users, as if their failure to add up was somehow a failure of the tool itself.

    The problem is exacerbated by the use of  segmentation or filtering, because then you find that (No. of users who did A) + (No. of users who did B) > (Total no. of users), because, of course, some users did both A and B.

Some low-end tools sidestep both these problems (at the expense of their credibility) by calculating daily UU numbers and then just adding them up for the weeks, months numbers etc; and by not offering any segmentation capability. So poorly educated users don’t see numbers that confuse them, and the tool doesn’t have to go to the trouble of calculating UU numbers properly. But those tools are a dying breed.

Another way around the challenges of providing UU numbers (which has more integrity than just calculating them badly) is to avoid providing them at all, and instead to convince your users that what they really need to measure is visit (or session) numbers to measure the effectiveness of online marketing.

Eric Peterson has an interesting post on his blog where he quotes an attendee at the recent E-metrics Summit, who denounces the attitude that visit-based conversion rate calculations are the best as “crap”. So there’s clearly still a lot of debate about whether visit or UU (visitor) numbers are better. I tend to agree with Eric’s assessment – that you should use both for different reasons. I’ll address this topic in more detail in a future post.

2 thoughts on “Are UU numbers worth the bother?”

  1. Ian,
    The problem I have with even beginning to debate whether visitor (unique visitor, unique user, etc.) counts have any value or should be used arises from the fact that is is unreasonable to try and get any organization to “look the other way” when they ask you about the number of people who come to the web site.
    Yes, I know that cookies are not people. Trust me, I know.
    But once you get past the relatively simple explanation about the differences between “cookied browsers” and “people”, in my experience, business owners will nod their heads and say, “Okay, we get it. So how many people came to our web site in the last 8 months?”
    I remember when Belkin first wrote his posts on the subject, I called a number of my friends who happen to be among Omniture’s biggest customers and asked them what they thought. The responses ranged between “Wow, I’m not sure it would go over very well to stop reporting visitor counts …” and some more colorful things I would never repeat in print for fear my children would read it someday.
    Visitor counting is part of what we do as web analytics professionals. Counting and estimating. Counting, estimating and reconciling as necessary. That’s why, when I was at JupiterResearch, I hammered home the idea that cookie deletion is a fact of life and so the best thing to do is work to figure out how it affects your web site and to factor that into any reporting you do on visitors.
    Some people have taken this advice. Some have not. Some will never take it because the calculations require some set-up and forethought. C’est la vie, eh?
    Anyway, I’m sorry to hear you’re having to debate the value of visitor counts in your organization. Not all applications struggle to generate visitor counts for odd or unique timeframes. I’d be happy to show you the approach I use to count visitors if you’re interested.
    You know where to find me.
    Eric T. Peterson

  2. Eric,
    Thanks for your long comment!
    We’re not debating the value of UU counts in of themselves – it’s just that there is a cost to delivering them (however elegantly the tool does it), and at the unsophisticated end of the market, it can seem a little thankless because a lot of the time the user feedback is tilted more towards suspicious incomprehension of these “numbers that don’t add up” rather than joy at being able to see the number of people (or at least, unique cookies) doing a certain thing.
    And I can see the logic of Belkin’s position, if not exactly for the reasons he states – there’s an argument that says that it may be more worthwhile in certain circumstances to provide a less useful metric that is better understood than a more userful metric that is poorly understood.
    As for the cookie churn thing – well, you’re right, of course. That’s a whole other can of worms.

Comments are closed.