A bit late to the punch with this one (as ever), but Comscore have published a white paper which contains more detail about the methodology behind their recent study which showed that both first and third-party cookies are deleted more than we thought (or at least more than we hoped). It clears a few things up about their claims; and raises some interesting new questions.
Log-in vs ‘passive’ cookies
The Comscore study used ‘passive’ (non-login) cookies from Yahoo! for its first-party cookie. As per my previous post on this topic, the high deletion rate of this cookie (31% at least once a month, according to Comscore) reiterates the need to use login information (i.e. cookies) for UU counting where possible.
Cookie ‘flip-flop’ vs cookie deletion
Buried in the numbers on page 8 of the report is a footnote about ‘preserved’ cookies. According to Comscore:
“Preserved designation includes PCs where two or more distinct cookie values were observed alternating throughout the observation period. Such oscillating patterns reflect the use of multiple browsers, or multiple accounts on a PC, and do not reflect reset events.”
From the study data, 69.3% of first-party cookies were either constant for the month of the study, or ‘preserved’ according to the above definition. Another 16.1% were reset once (meaning that there were two distinct cookie values during the period). But in order to tell whether a cookie value change is due to deletion or multiple accounts, you have to have at least three cookie values: A, B, A. I’m going to call this a ‘recurring’ cookie value. So there’s a good chance that a proportion of the “1 reset” group is actually recurring cookies where there was only an A, B pattern (i.e. the recurring cookie didn’t come back again before the month was up).
The report isn’t clear about whether the study stripped out recurring cookies from the reset counts in higher groups. For example, if they saw the following cookie values through the month:
A B C D
then that is clearly three resets. But if they saw the following values:
A B A D
then is that three resets, or did they strip out the extra A and call it two resets?
This is important because one of the banner headlines from the study is that 7.1% of computers contribute 36.3% of the unique cookie values, resulting in an average overstatement of UUs of 150%. If actually these numbers haven’t had recurring cookies stripped out, then the overstatement wouldn’t be that high.
Cookie awareness
One interesting aspect of the study which wasn’t in the original press release was the results of some survey questions that Comscore asked. The most interesting one was “Do you know the difference between a first-party and third-party cookie?” An astonishing 29.8% said they did – I’m not sure even that many people here know the difference. Only 4.2% claimed to selectively delete third-party (but not first-party) cookies, however, which is unsurprising (or surprisingly high), since it’s basically impossible to distinguish between first- and third-party cookies once they’re on your system, unless you keep a database of the sites which are known to issue third-party cookies.
Overall, the study makes for interesting reading, and seems to have been undertaken with some care. However, towards the end Comscore throws in a bunch of other reasons (rotating IP address, accidentally including international numbers when comparing with domestic panel data) which also inflate server-based counting methods. The addition of this extra material simply has the effect (for me, at least) of making the report seem even more self-serving – it seems disingenuous to release a paper (written in however scholarly a fashion) which basically just bashes server-side measurement when Comscore’s motives are so easy to see. It contributes to the debate, and I welcome it, but it leaves a nasty taste in my mouth. Not something that can be said for the splendid Father’s Day gift that I received on Sunday (my family know me too well).