The Marketing Accountability Standards Board (MASB) invites marketers to participate in a new research initiative on targeting accuracy and return on objectives and investment.
WHY THIS IS A PRIORITY
There is an urgent need to take a hard look at current practices and quantify the financial implications of out-of-segment placement of digital advertising. Preliminary research from MASB and Truthset across 16 data providers demonstrated substantial disagreement in demographic profiles used for campaign segmentation. This will only get worse with the phasing out of third-party cookies, expansion of commercially sponsored consumer anonymity programs like Apple ATT, and data-sharing evasion by younger generations.
MASB and Truthset are partnering again to conduct very rigorous experiments with data from recently and soon-to-be completed client campaigns to quantify targeting accuracy and the resulting impact on marketing objectives and financial returns. Participating marketers will receive the results for their own targeted campaigns and aggregated comparison benchmarks from all participants. This will provide understanding of the upside in efficiency and effectiveness from adopting targeting best practices.
The objective is to equip marketers with this valuable information within the 2023 calendar year to inform campaign planning going into 2024. Onboarding meetings are currently underway.
WHAT WE PLAN TO DO
The need for reaching specific consumer segments with focused messages has never been greater. Marketers have gravitated towards digital media channels as the primary way to implement personalized marketing and have been benefiting from the resulting financial returns. However, the industry’s main vehicles for delivering segmented audiences have recently been facing substantial headwinds.
Privacy concerns have driven consumers to tighten their sharing of personal data. In a recent study by market research firm Pantheon, 73% of consumers reported a willingness to share information but this drops by generation to only 35% for Gen Z. Furthermore, 48% of consumers reported evading sharing data by using guest checkouts, while 42% declined to create a user profile. [1]
So, even first-party data sources can be incomplete and misleading. Add to this the decline – and soon disappearance – of third-party cookies, and the accuracy of targeting solutions comes into question. This leads to several key questions:
- Generally, how accurate is our industry’s current ability to target segments in digital channels?
- What impact does this have on marketing returns for digital campaigns?
- How would this improve with the adoption of targeting best practices?
MASB began investigating this issue in 2018 as part of its Digital Accountability Measurement project. In a poll of practitioners at the 2020 MASB Summer Summit, [2] targeting and data integrity were identified as top challenges facing marketers in managing their digital investment. In response, MASB partnered with marketing research firm Truthset on a first-of-its-kind preliminary assessment of segmentation consistency. Truthset’s mission is to measure the accuracy of record-level consumer data so that companies can make more informed decisions to produce better business outcomes.
In the research, gender, age range, and other basic demographics for a large sample of consumers were drawn from 16 leading digital targeting data service providers. Shockingly, agreement on gender among them was only 51%. For age ranges, where more options were available, the agreement was only 7%.3 This initial investigation demonstrated that there is a great disparity in segmentation groupings generated by different suppliers utilizing basic demographics, let alone the more complex behavioral, lifestyle, life stage, and interest attributes.
Based on this, there is an urgent need to quantify this inaccuracy and to understand its impact on marketing return for individual digital campaigns. This will provide marketers with the upside potential for adopting segmentation best practices. We hypothesize that the best alternatives will have the potential to deliver more efficient and effective campaigns and deliver substantially greater monetary returns than marketers receive today.
We are now recruiting Marketers to participate in a MASB-sponsored study to do this.
The TARGETING ACCURACY & ROI IMPROVEMENT EXPERIMENT
MASB and Truthset are once again partnering to further dissect the level of accuracy of data sets used for demographic targeted media investment and quantify for the first time its impact on ROO and ROI.
Whereas the preliminary research focused on agreement in basic demographics across 16 data service providers, this new work will draw upon an expanded 20 for segmentation. While the earlier work looked at general agreement across the providers, this research will calculate the practical accuracy using data from recently completed in-market digital campaigns of participating marketers.
The next step will be a privacy-protected cleaning of the marketers’ campaign segments using Truthset’s cross-provider data accuracy scoring technique to exclude records containing demonstrably incorrect targeting information. We will then recalculate returns in ROO and ROI lift studies across all participating marketers’ campaigns based on the restated data. This will establish a clear benchmark for the improvement, in terms of both the marketer’s campaign objectives and financials, from adopting more accurate segmentation techniques.
Marketers who participate in this research initiative will receive the results for their own targeted campaigns and aggregated comparison benchmarks from all participants (expected to include over a dozen brands from three to four marketers).
Except as approved by the individual participating marketers, only aggregated or blinded data will be released publicly.
PROCESS and DATA NEEDS
MASB and Truthset will process all data under NDAs with marketers.
Marketers, through their designated agencies, will provide the log files containing the post-exposure impressions for their campaigns and the corresponding ROO and ROI lift data. This can be provided concurrent with the campaigns’ live runs or can be done retrospectively (within several months). They will also provide the definitions used to place individuals into the target segments. Campaigns will need a minimum of 50 million impressions across two months or more to be of sufficient size for the analysis.
Truthset will use these log files to pull the corresponding individual-level data from the data collective of 20 segmentation data service providers. Truthset will then determine the likelihood of each truly being in the specific segment. From this, an estimate of the practical accuracy of the existing targeting will be calculated.
This placement will then be applied to the corresponding lift data in an A/B test approach. One cell will include those determined to be in-segment via the approach while the other will include those determined not to be in-segment. This will give the relative performance of in-segment and out-of-segment individuals.
This new in-segment cell (that is, cell with inaccurate records removed) will also be compared to the original to show the resultant difference in return of adopting the enhanced technique versus the existing approach.
Further cuts of the data may be used to control other variables, for example, filtering on initial impressions to control for campaign wearout across varying campaigns. This will be determined as the work proceeds and will be included in reports to provide additional learning.
Also, if control levels for a marketer’s ROO/ROI are calculated via individual-level data that is not included in the exposure log file, an estimated adjustment based on median results may be applied to the original control. This will help minimize overestimation of improvement from out-of-segment individuals remaining in the control.
TIMING
While this experiment will be dependent on the timing of the campaigns, we anticipate completing the analysis in the 2023 calendar year. Reports will be delivered to participating marketers at least one month prior to public announcement of aggregated and blinded results.
PARTICIPATE!
We are looking for three to six marketers take part in the experiment. Since the analysis is based on campaigns, participants with multiple brands are encouraged to include campaigns for each advertised brand. Single brand marketers can also provide multiple campaigns if more than one is in effect during this project’s duration.
To schedule a meeting to discuss participation, please contact MASB Executive Director Frank Findley, frankfindley@themasb.org, or Truthset Vice President of Client Partnerships Wes Waterston, wes@truthset.io.
FOOTNOTES
1. Media Post, Consumers Prefer Generic Messages to Personalized Ones, Study Finds. January 31, 2023, mediapost.com/publications/article/382043/
2. Karie Ford, MASB Digital Accountability Measurement: The continuing evolution of digital marketing, Reported MASB 2021 Spring Summit. April 8, 2021.
3. Kelly Barret, Truthset and Joe Plummer, Sunstar; Digital Marketing Accountability: Targeting, November 11, 2021.