The Wrong Metrics
28 February 2024
This post is made up of elements that have appeared previously on ‘The Media Leader’ in two separate pieces in February 2024
For some while now the industry has faced a problem with its basic metrics. I say ‘faced’ although the truth is we haven’t faced up to the issue at all, doing that curious media dance of sticking our fingers in our ears, twirling round three times, clicking our heels, and hoping the problem will go away.
Take share-of-voice, or SoV.
Understanding competitors’ creative and media activities is a standard feature of every post-campaign evaluation.
Beyond these, SoV is often considered an input to plans.
I’ve never understood this. Planners don’t benefit from magical foresight into competitive plans, so aiming to achieve a certain SoV relies on guesswork.
Plus, SoV is based on available adex or analysable audience data. That made sense when most major brands’ spend was on TV or press where expenditure and exposure data were reliably reported across the industry.
But that is no longer the case. Owned and retail media forms are growing, as are online channels where data is limited / non-existent.
So, share-of-voice is based on data from media forms that no longer represent the majority of a brand’s communication activities. Drawing conclusions from it can be highly misleading.
It makes more sense to examine activity from the other end of the telescope; moving from what’s transmitted to what’s received.
The IPA has recognized the importance of owned media and has worked with MESH Experience on why owned channels are important and how to create tools to measure them.
This allows brand owners to assess the relative importance of all communication channels – from paid-for advertising through retail media to owned websites, vehicles or clothing.
MESH has created the Experience Impact Score (EIS) for all brands within a category. EIS uses data on what’s received, and what’s noticed providing planners with something complimentary to impressions-based audience data.
A classic use case for SoV is in setting budgets.
Back in the last century, Carat created a tool called ‘Budget Bearings’, the brainchild of their resident genius, Phil Gullen. The idea was to gather as many pieces of information relevant to the task of budget setting as possible and plot them within the tool. The more information, or points of reference the better in plotting an accurate position.
Budget Bearings relied primarily on output measures, calculated from available data, like share-of-voice.
Marketing then was seen as a cost with few long-term business success metrics attached to it.
Slowly we’re starting to win the argument that building brands through marketing and advertising is an investment, not a cost.
This shift should be reflected in the way budgets are set.
Setting budgets around an output metric like reach or SoV is no longer sufficient at a time when we have access to so much more.
Basing financial recommendations around concepts the industry either does not measure across all channels or can’t define makes no sense.
We need to learn FD-speak, and to be precise and consistent. Most FDs are neither familiar nor impressed by the metrics used within planning and buying.
We buy audiences because the number seeing and noticing the advertising is connected to financial results.
But audience data is a stepping stone to the destination, not the destination itself.
We should make more use of modelling, taking in all communication channels, advertising and non-advertising (given the upside that marketing budgets are by definition larger than ad budgets).
My Crater Lake colleague, Co-Founder of the analytics business NavigationME, David Beaton makes the point that in over 20 years building brand models, most underspend against their market opportunity.
David believes this is because of a lack of reliable, accurate, consistent tools not only to measure this underspend, but also to predict and optimize the effect of any increases.
Many econometric and market mix models limit themselves to paid-for media forms and look backwards, using limited inputs. Many don’t consider consumer research data, as if how consumers think has no impact on what they eventually do.
David’s advice to anyone considering modelling to justify budgets is:
- Build models that cover a holistic list of factors that drive or suppress sales. Not only paid media, but also earned and owned; and baseline factors important to the category and brand.
- Any statistical model has to demonstrate accuracy by aiming to explain over 90% of variation in sales over the last three years. This will help persuade the CEO and FD and reduce the risk of misattribution and poor outcomes.
- The model must be able to use optimization and simulation to estimate the gains to be had from budget increases. These models typically yield a diminishing return curve; as spend increases, incremental sales lift falls. At some point, any further spend increase yields no sales increase at all; this is where marginal benefit equals marginal cost. Given the current situation, this is the maximum level of sales that marketing communications can drive. This is not the same as ‘maximizing marketing RoI’. In optimized campaigns marketing RoI falls as spend increases, so maximum RoI happens at low spend levels; chasing marketing RoI leads to lower budgets.
- Combine both prediction and optimization at different spend levels. As spend increases, the channel mix should change, as different channels have different effect curves.
- Test alternatives. Work your way up the spend curve over time; controlling for both your own spend risk and factoring in likely competitor reaction.
Agencies like to believe they can do anything. But they’re hardly channel neutral. They see advertising as the solution to any question. If you call in a plumber, he’ll find you a leak.Predictive modelling is a specialist area, like creating great copy.
When all anyone had was output data, the agency could plot the position.
Now we have storms, choppy seas, and an under-equipped boat. It’s not the weather to be out with amateurs.