‘Our Industry Needs a Pivotal Change Towards Accurate Causal Measurement’, by Robin Davies, Country Manager UK, DE & APAC, Mediaplex

CPA is not a performance-based marketing metric. On the face of it, this may seem like an odd thing to say, but it is in fact a natural extension of the well-known adage “correlation is not causation”. With marketing analytics as such a hot topic, the discussion of these details is highly relevant. Brand investment into analytics is expected to rise by 80% within the next three years (Source: TheCMOSurvey.org), so if you’re a brand looking to make such an investment, it’s important that you understand the details of how accurate measurement is done (or hook-up with someone who does).
Whether post-click or post-impression, attributing your transactions or conversions to a particular media source, based on a set of manually derived attribution rules, is just that – manual and subjective. Regardless of what such solution vendors would tell you, the hard statistical truth is that fractional/manual attribution doesn’t actually measure the incremental effect of that media source on revenue (in fact, it doesn’t even measure true statistical correlation of ads to revenue). Fractional attribution rewards your vendor for showing up as much as possible – and everywhere in the funnel, not just at the end. It’s an anti-efficient approach to incentivising your vendors.
Unfortunately, the newer algorithmic solutions, based on sexily-named techniques, like “Bayesian regression” and “dose-response modelling”, don’t accomplish the goal of incremental measurement either – at least not on their own. This is not an opinion – these are hard statistical facts, as outlined in the many academic papers establishing the principles of causal inference. To get there, they would require additional experimental design (read: A/B testing) to be employed on top of that solution… in which case, what’s the real ROI on that solution over a good statistician and more basic statistical software?
Next time you evaluate your vendor activity, ask yourself this question – was it the media that caused the sale, or did the likelihood of a sale cause the media? True incremental measurement should be the gold standard for every advertiser – it’s the singularly correct economic outcome for your business to use in optimising its behaviours. There is academic literature available which sets out the requirements around how this can be accomplished – in plain terms, show correlation between treatment and effect, then rule out the possibility of any variables being the actual cause of both treatment and effect (known as “confounding”). The challenge, in more industrial terms, is then clear – we need to separate the effect of a targeted audience (over the more general population) from the effect of an ad on that audience. For example, the effectiveness of ads served to recent site visitors should really be measured by comparison to recent site visitors with the same profile who were not served the ad; otherwise, we cannot separate the value of an audience who have visited the site before from the value the ad is creating within that audience of previous site-visitors.
Digital media is generally being evaluated and justified using just correlation (revenue matched to the media investment via cookie correlation). This approach does not measure the cause of media spend on revenue; it just correlates returns to media investments and assumes causality according to some entirely arbitrary and subjective set of rules. Measuring ROI like this is just wrong and, once you understand this, there’s no going back. A savvy few marketers are challenging this status quo at the moment, and it feels like the next big pivot-point for our industry.
If brands care about the cause of media investment on revenue, then they need to drive for better understanding and adoption of the appropriate statistical techniques that will give a clearer picture of what’s working and what’s not. Brands are ready to make changes, they want regular, directional advice on channel and vendor profitability, along with specific and actionable instruction on what attributes to target, so vendors can become profitable, rather than just target cookies on users who are likely to buy anyway. Brands are the motivators for “doing better” not “doing more.” The vendor community thrives on “doing more” with more technology, more media – more everything. This is a natural consequence of the business environment of our industry.
The vendor and agency community that advises and supplies our industry with measurement technology in large part operates under commercial models that see them benefit from increased media budgets and volume – not increased efficiency. You see the challenge here.
A change in how brands measure digital media effectiveness would likely change everything within the industry. It could be very disruptive to the status quo, as Big G or any other significant media owner can’t be seen to support it as it risks revaluing media effectiveness, to their detriment. The brand community has recognised that measurement needs to change if the industry is to move forward, but solutions like causal measurement need to be championed in-house if they’re going to get any traction, as there are too many conflicting business objectives for this to come out of the vendor community. Brands will want to adopt this, but they’ll have to consider how to align motives in the brand-agency relationship. Brands that optimise towards profit would love to get the same revenue for half the media spend, but any agency partners paid as a percentage of spend will be less than happy about this. Brands need to allow their agency partners to profit also, assuming they still have an effective role in achieving that brand profit. However, vendors have little interest in disrupting the profitable status quo in which so many vendors are now unwittingly complicit. It’s not the fault of vendors that brands have been paying their vendors to spend their budgets this way.
Changing to incremental value, or causality-based measurement, is not just a different approach to measurement, it’s a paradigm shift for the marketing industry. I anticipate a pivotal change in digital marketing now that brands are starting to take control and demonstrate the significant difference in return on ad spend that can be achieved once correct or causal measurement is established. Brands will be in the enviable position of choosing how to reinvest the media spend that was being wasted; either to drive more digital sales in a truly incremental way or to further the in-sourcing of services and up-skilling of their teams. The brand-side thought leaders that champion this shift will be able to demonstrate that they were adopting causal measurement from the moment it became available and the followers can make their excuses as they see fit. The spotlight is on brands now to think for themselves and embrace this opportunity. Anyone who loves digital has to be excited by this.