SPECIMEN · Marketing · · 1 min read
Patterns Across a Few Hundred Thousand Ads
I analysed a few hundred thousand ads trying to extract real patterns and mostly learned why the question is harder than the industry admits.
The brief was simple: take every ad we could get our hands on, find what the winners have in common, and write it down. A few hundred thousand creatives later, the honest answer is that the winners have less in common than anyone selling a framework would like.
This is not a story about a clever model. It is a story about how a question that sounds empirical turns out to be mostly a question about what you are willing to count as evidence.
What we actually had
Creative, spend, impressions, clicks, and the platform's own idea of a conversion. That last one is the problem. Every platform claims credit for the same purchase, and none of them will tell you the truth, so any pattern you find in "conversions" is partly a pattern in who got to report first.
The first pass
Cluster the creatives, regress performance on cluster membership, read off the coefficients. It produced a beautiful deck. It also produced results that reversed sign when we changed the attribution window from seven days to one.
A method that never gives you an answer you didn't want hasn't really been tested yet.
What survived
- Novelty beats polish, briefly. Every format decays, and the half-life is shorter than the production cycle.
- Nothing about "psychological triggers" was distinguishable from noise once we controlled for who was shown the ad.
- The single strongest predictor of an ad's reported performance was the platform reporting it.
Why this matters
Because once you take attribution seriously you stop being able to accept most of what marketing tells itself. The gap between what you can measure and what you can claim is the whole field, and it is wider than the dashboards suggest.
I still run paid campaigns. I just no longer believe the report.