What paid social was actually worth

The insurer was spending tens of millions a year across paid media, and paid social reported the strongest return of any channel by a wide margin. On the strength of those platform-reported numbers, it kept getting more budget.

The problem was that finance never believed them. The platform's own dashboard was both the salesperson and the scorekeeper, and the return it claimed didn't reconcile with what showed up in actual policy sales. Two parts of the same business were running on two different truths, and the bigger the spend got, the more that gap mattered.

Last-click and platform attribution answer "who touched this conversion last," not "would this conversion have happened anyway." Retargeting is especially flattering here: it serves ads to people already heading for checkout, then takes the credit when they arrive.

So the question wasn't "how many conversions did paid social get credited with." It was how many of those conversions wouldn't have happened without it: the incremental ones. That's the only number worth setting a budget on.

The cleanest way to know what advertising caused is to switch it off somewhere comparable and watch what happens. So the core of this was a matched-market geo holdout: turn paid social off in a set of regions, keep it running in a matched set, and read the difference.

Markets were paired on baseline conversion rate, size and seasonality, then split. To avoid trusting a single method, I triangulated the bottom-up experiment against a top-down marketing mix model. When an experiment and an MMM independently land in the same place, you can believe the answer.

Treated regions did outperform held-out ones, so paid social was genuinely doing something. But the gap was far narrower than the platform's numbers implied: only about a third of attributed conversions were incremental, and the effect sat comfortably above the test's detection threshold rather than anywhere near the reported figure.

Reading incrementality channel by channel changed the picture entirely. Paid social was the most over-credited; non-brand search and video were carrying more real weight than their attributed numbers suggested. The reallocation followed directly from that.

The immediate result was a +27% lift in blended incremental ROAS: same total spend, just pointed at what actually moved sales. But the lasting change was structural: incremental return, not platform-reported return, became the metric the media budget is built on.

That framework now governs roughly $42M in annual media spend, run as an always-on cadence of holdouts rather than a single test. And the two-sources-of-truth argument with finance simply went away, because both sides were finally looking at the same defensible number.