How to Measure Marketing Impact With Incrementality Testing

Facing intense competition and increasingly tighter budgets, businesses across all industries have been looking for ways to optimize their ad spend and maximize their ROI. To determine which campaigns work and which don’t, solid marketing measurement tools are not optional, but a must. Indeed, modern marketing is facing major transformations: with the gradual fade-out of third-party cookies and multiplying touchpoints both on and offline, one cannot rely solely on attribution. Designed to assign credit to different touchpoints based on user behavior, attribution was the go-to model for many years, thriving on the then-wide accessibility of third-party cookies. But with limiting factors such as their deprecation on several major browsers and increased regulation, such as RGPD, this once-ubiquitous model might not be enough anymore.

To circumvent this, marketers find themselves completing their measurement approach with Marketing Mix Modeling, or MMM, and incrementality testing. This methodology helps marketers determine whether their campaigns are what is driving customer interest, or if demand would have been the same without them.

For a deep dive on marketing measurement in audio form, listen to our Data Break podcast episode with fifty-five UK’s Head of Media Nick Yang and Funnel’s VP of Measurement János Moldvay. The following article contains insights explored in more detail throughout the episode (French version entirely dedicated to incrementality testing available here).

What is Incrementality Testing?

Incrementality testing is a methodology designed to measure the causal impact of marketing initiatives through controlled experiments. 

While attribution determines which touchpoint of the user journey led to a conversion, incrementality testing pinpoints the real effect of marketing by comparing a group exposed to an ad (test group) with a group that is not (control group).

For example, out of 1,000 conversions, how many were driven by an ad campaign? How many customers would have made a purchase regardless of marketing actions? Incrementality testing helps answer these exact questions.

Why Incrementality Testing Matters

While traditional attribution models (such as last-click or multi-touch attribution) are growing less reliable, incrementality testing provides a clearer picture of actual marketing effectiveness by:

• Avoiding Over-Attribution – With attribution, advertising platforms often take credit for conversions that would have happened anyway. Incrementality testing helps remove this bias.
• Optimizing Budget Allocation – By identifying which campaigns, channels, and strategies are actually driving value, marketers can reduce wasted ad spend.
• Measuring Cross-Channel Impact – By turning one or several channels “off,” incrementality testing helps businesses understand how different marketing channels work together to influence consumer behavior.

How Incrementality Testing Works

To conduct an incrementality test, marketers must follow a structured approach:

1. Define the key metric you want to measure, whether it’s conversions, revenue, app installs, or any other KPI to ensure clarity when interpreting results.
2. Create randomly segmented Test & Control Groups

- Test Group: individuals who will be exposed to the marketing campaign

- Control Group (or Holdout Group): Individuals who will not be exposed to the campaign

3. Run the marketing campaign while ensuring that only the test group is exposed to the ad while the control group remains unexposed.
4. Measure the Results after some time has passed by comparing the conversion rates between test and control groups. The difference between the two rates represents the incremental lift, or the percentage of conversions that were driven solely by the campaign.

Challenges of Incrementality Testing

While incrementality testing is powerful, this methodology is not without challenges:

Data Volume & Statistical Significance – Small sample sizes can produce unreliable results. Select a large enough audience and adjust test duration to ensure that the results obtained are relevant.

External Variables – As many factors can affect test results, from market shifts to competitor actions and seasonality, they must be accounted for in the analysis process.

Cost & Implementation – Some businesses hesitate to use holdout groups because it means deliberately excluding potential customers from a campaign. Still, the long-term benefits of better budget allocation often outweigh short-term losses.

fifty-five's Best Practices for Incrementality Testing

Here are fifty-five’s best practices to maximize the impact of incrementality testing:

• Run Your Tests Across Different Channels – Test different platforms (search, social media, email, display ads) to find out which marketing mix is the most efficient.
• Use Multiple Measurement Methods – Complement incrementality testing with other measurement techniques, such as Marketing Mix Modeling (MMM) and A/B testing, for a more holistic view of performance.
• Continuously Refine Your Campaigns – Remember that incrementality testing should be an ongoing process, not a one-time experiment. Run your tests regularly and refine your marketing strategies for actual long-term success.

Final Thoughts

Incrementality testing can be an invaluable tool for brands looking to understand how effective their advertising efforts truly are. By measuring the real impact of marketing actions, businesses can base their decisions on actual data, optimize their spending strategy, and improve campaign performance. If you would like to start testing and uncover the true ROI of your marketing efforts, don’t hesitate to get in touch with us! And to discover how fifty-five combined a custom, in-house MMM with incrementality testing to help TotalEnergies save +$4M, read our case study.