It’s not the test it’s how you run it
🥲 Your creative tests keep coming back inconclusive. The tests aren't broken. They're too small to see anything, Media buyer index of the week, and more!
Howdy readers 🥰
In this newsletter, you’ll find:
🥲 Your creative tests keep coming back inconclusive. The tests aren't broken. They're too small to see anything.
📊 CAC Improved Across the Board, But ROAS Is Telling a Different Story on the Biggest Platforms
🏆 Ad of the Day
If you’re new to ScaleUP, then a hearty welcome! You and 50k+ CEOs, CMOS, and marketers have reached the right place. Let’s get into it, shall we? Oh! Before you forget, if someone forwarded this newsletter to you, don't forget to subscribe to our newsletter so you never miss out!
Together with AirOps
Google I/O Just Changed Search. This Session Breaks It Down.
Your rankings look fine. Your traffic is quietly disappearing.
AI Mode no longer sends clicks. It just answers, and whoever gets cited wins the buyer.
Chris Long co-founded Nectiv and has reverse-engineered AI citation patterns across hundreds of B2B content audits. He knows exactly why some content surfaces in AI answers while identical pages sitting above it in rankings get nothing.
On June 17th, he's breaking it down live in 60 minutes:
- The content signals AI Mode actually uses to decide what gets cited
- How to audit your existing pages for AI visibility gaps before Q3
- The refresh workflow that moves content from ignored to referenced
- How to measure real search presence when rankings stop telling the truth
The old rules are gone. This session covers what replaced them.
Can't make it live? Register anyway and get the recording in your inbox within 24 hours.
🥲 Your creative tests keep coming back inconclusive. The tests aren't broken. They're too small to see anything.
Every creative test answers one question, and only if one variable changes, everything else holds constant, and the variant count is high enough to mean something. Most testing violates all three: each new ad swaps the creator, the script, the framing, and the hook at once, and only 3-5 variants get produced.
The Statistical Floor Nobody Runs The Math On
Detecting a 20% lift in CTR or CVR at 95% confidence requires roughly 15-25 variants per variable.
A test with 3-5 variants can only detect lifts of 50% or more, which rarely happen. The 10-20% improvements that actually compound program performance over time are statistically invisible at the counts most teams run.
So the test does not say "nothing works." It says "no detectable winner." The team reads that as creative failing to move the needle, when the needle moved, and the test could not see it.
The Fix: Hold The Creator Constant, Run 15-25 Variants
Same person, same setting, same tone. Change only the content variable:
- Different hooks on the same scene
- Different framings of the same message
- Different first-frame approaches on the same script
The variant count is what makes it work. At 15-25, the test detects mid-sized effects. At 3-5, it detects nothing short of a landslide.
Insense makes this volume cost-effective: one brief, 20+ raw footage clips to mix and match into scroll-stopping variants, lifetime usage rights, delivered in 14 days or less. Exactly the raw material a properly powered content test requires. You can book a free strategy call and get $200 for your first campaign.
Test Content Before Persona, Not The Other Way Around
The sequence is not arbitrary. Persona effects depend on content effects.
A punchy hook lands through one creator and dies through another, which means any persona finding produced before the content is locked gets invalidated the moment the content changes underneath it.
Lock content first through creator-constant testing. Once you know which hooks, framings, and first frames actually work, run persona tests using the winning content. Findings produced in that order survive the next round of testing instead of being quietly overwritten by it.
Underpowered, multi-variable, out-of-sequence tests teach nothing. Properly powered, isolated, sequenced tests teach something every time.
📊 CAC Improved Across the Board But ROAS Is Telling a Different Story on the Biggest Platforms
Last week's Media buyer index looked efficient on the surface; acquisition costs fell almost universally, but return quality fractured underneath, with several high-spend platforms posting CAC gains while ROAS moved in the opposite direction.
The Breakdown:
CPC - YouTube and Axon saw click costs fall while TikTok and Snapchat pushed higher, on channels where CPC rose, but CAC still improved like TikTok, don't cut spend; instead, refresh creative to protect conversion rates before costs climb further.
CAC - Every platform improved acquisition costs this week, but where CAC gains aren't backed by ROAS improvement like Meta and TikTok, run a mid-week performance check before increasing budgets, as the efficiency may not survive higher spend pressure.
ROAS - Pinterest led at +13.99% and Axon posted +12.55% while Snapchat (-7.27%) and TikTok (-1.99%) declined, move fresh creative into Pinterest and Axon now, cap Snapchat spend at current levels, and treat TikTok's CvR strength as a post-click optimization opportunity.
Meta holds 54.61% of spend with improving CAC but -1.10% ROAS, a warning sign at that scale. Pinterest at 0.57% and Axon at 1.79% are posting the two strongest ROAS figures on the board while sitting at near-zero budget share. Reallocate 10–15% of Meta's incremental growth budget into these two channels this week and measure CAC at the new spend level before the next cycle closes.
🎥 Ad of the Day
What Works:
Consequence Selling - The ad highlights the cost of the condition, not the condition itself. Consumers care more about losing sleep than understanding eczema mechanisms.
Problem Visualization - The ointment physically covers part of the word "sleepless." The product visually demonstrates its function instead of explaining it.
Cognitive Minimalism - Nearly the entire message is communicated in three words. Low processing effort makes the ad highly effective during distracted scrolling.
Don't advertise what your product fixes. Advertise what consumers get back once the problem disappears. People rarely buy eczema cream, sleep aids, or pain relief products because of ingredients. They buy better mornings, better sleep, and better days.
🥳Events
🔥 The New Search Signals That Matter More Than Rankings
June 17 | 60 Minutes | Virtual Event
Your rankings can hold steady while your visibility declines. Learn how AI search evaluates content, what drives references in decision-stage queries, and the refresh frameworks helping marketers adapt to Google's biggest search shift in decades. Chris Long, VP of Marketing at Nectiv, breaks down what belongs on your roadmap now.
Can't make it live? Register anyway, you'll get the recordings within 24hours.
Advertise with Us
Wanna put out your message in front of over 50,000 best marketers and decision makers?
We are concerned about everything DTC and its winning strategies. If you liked what you read, why not join the 50k+ marketers from 13k+ DTC brands who have already subscribed? Just follow this.
At ScaleUP, we care about our readers and want to provide the best possible experience. That's why we always look for ways to improve our content and connect with our audience. If you'd like to stay in touch, be sure to follow us EVERYWHERE🥰
Thanks for your support :) We'll be back again with more such content 🥳