How To A/B Test Ad Placements to Maximize Revenue Without Hurting Retention

How To A/B Test Ad Placements to Maximize Revenue Without Hurting Retention

Most apps that run ads pick a placement, set it up, and never revisit the decision. A single ad placement that generates $2,000/month might generate $3,500/month with better placement — and it might have better retention too, because the placement is less intrusive.

This is why A/B testing ad placements matters. Here is how to do it properly.

What to Test

Ad optimization A/B tests typically focus on:

Placement position:

Ad format:

Timing/frequency:

Consent and context:

How to Set Up the Test

A proper A/B test for ad placements requires:

1. A single variable per test. Do not change position AND format at the same time. You will not know which change drove the result.

2. A clear primary metric. For ad tests, this is usually one of:

3. A guardrail metric. This is the thing you do not want to break. Common guardrail metrics for ad tests:

If the winning variant improves eCPM but reduces 7-day retention, it is not a win. Revenue from a churned user is zero.

4. Sufficient sample size. Run each variant until you have at least 500-1,000 users in each group, and at least 7 days of data (to capture both weekday and weekend behavior patterns).

What the Data Usually Shows

From ad placement A/B tests across apps:

Placements below content consistently outperform placements above content on retention metrics, while performing within 10-20% on revenue. Users who see an ad after completing their task are less annoyed than users who see it before.

Native card formats typically outperform banner formats on CTR (click-through rate) by 2-4x, which drives higher eCPM. The CPM is higher because the clicks are more qualified.

Placement with brief explanatory context ("Ads keep this app free for everyone") reduces negative sentiment in user surveys without significantly impacting revenue metrics.

Every-session frequency vs. every-other-session typically shows 20-40% higher revenue from every-session, but 3-8% lower 7-day retention. The LTV calculation (lifetime ad revenue vs. lifetime of the user) usually favors every-other-session for apps with strong network effects or paid conversion funnels.

Measuring the Tests in Zerocost

Zerocost's analytics dashboard segments performance by placement ID. To run an A/B test:

  1. Create two placement IDs in the Zerocost dashboard (variant A and variant B)
  2. Use your app's A/B testing infrastructure to randomly assign users to each variant
  3. Log which variant each user sees
  4. After the test period, compare eCPM, impression volume, and cross-reference with your retention data

This gives you a clean comparison with Zerocost handling the ad serving and reporting for each placement independently.

Applying the Wins

Once a winning variant is identified (statistically significant improvement in the primary metric, no statistically significant harm to the guardrail metric), deploy it to 100% of users.

Then start the next test. Ad placement optimization is iterative — there is almost always another variable worth testing. A typical roadmap:

Four tests, each running 2-3 weeks, optimizes placement over roughly three months. The cumulative improvement is typically 40-100% higher revenue than the original placement — from the same user base, with the same or better retention.

Last updated: September 2026

ZerocostZerocost