You upload two creative variants to a Moloco campaign. At the next review, one has received more delivery and shows a better reported acquisition cost. The buyer needs to decide what to keep running; the creative team wants to know which idea to produce again. Those decisions require different levels of evidence.
Quick take: Treat a Moloco creative comparison as an observation unless you can verify the experiment’s assignment method, measurement basis and analysis plan. Adaptive delivery can support a practical decision to retain an asset, but a ranking alone does not establish that the creative caused the difference. Equal spend would not establish that either.
Moloco creative comparison validity depends on how the results were produced. This is one specific case of paid UA confounding factors: when delivery and creative selection evolve together, the asset is only part of the explanation.
How adaptive delivery changes the comparison
The immediate business implication is that a promising asset can be worth retaining without becoming a proven production formula. If you turn every delivery ranking into a claim about the winning hook, future briefs may repeat an explanation that the campaign never tested.
In its description of campaign optimization, Moloco says its AI evaluates and bids on individual impressions, selects relevant creative in real time and continuously learns from outcomes. That description gives no basis for assuming that two uploaded variants receive random, equivalent opportunities.
The resulting report describes performance under the delivery process that actually occurred. The users selected, timing of exposure and evolving optimization can differ between variants. Comparing a rate instead of a total helps account for volume, but it does not automatically remove those differences.

Unequal delivery is not, by itself, proof that an experiment is invalid. An experiment can use unequal assignment probabilities, and an adaptive experiment can be analyzed with methods appropriate to its design. The question is whether that design and its analysis are documented for your comparison. A dashboard with two creative rows does not answer it.
Separate an operating choice from a causal conclusion
Before reviewing the numbers, write down which decision you need. Choosing an asset for continued delivery is a narrower task than establishing that its message would outperform another message under comparable conditions.
- Descriptive observation: one variant has a better reported result within a stated period, scope and measurement definition.
- Operational preference: the available evidence supports keeping or prioritizing a variant under current conditions, with a review plan and explicit uncertainty.
- Causal conclusion: a documented experiment and suitable analysis support attributing a difference to the tested change, within the experiment’s scope.
An operational preference should remain conditional. It does not prove that the same asset will maintain its efficiency at a different spend level, or that its opening alone explains the result. If the variants also differ in length, copy and ending, even a valid comparison of the complete assets would not isolate the opening.
Moloco’s user acquisition page states that it supports A/B testing and incrementality testing. It also describes automatically testing creative variations. These statements establish advertised capabilities; they do not identify the assignment method used in your campaign. Ask for the actual test specification before labeling a report an A/B test.
Incrementality evidence also needs the right scope. A test of advertising versus a counterfactual without that advertising does not automatically answer whether creative A is better than creative B. The tested intervention must match the conclusion.
Check what qualifies the comparison
Use the following table before turning a ranking into a reusable creative claim. Have the buyer supply configuration and change records, and the analyst identify measurement definitions and available data. For each row, record the supporting evidence or mark it unknown. These are editorial review checks, not verified Moloco setup instructions.
| Check | Evidence to confirm | If unresolved |
|---|---|---|
| Tested change | Exact asset versions and every difference between them; whether the question concerns a hook, a whole asset or a delivery strategy. | Do not attribute the result to one creative element. |
| Assignment | What was assigned, how assignment occurred, its probabilities and whether those probabilities changed. | Keep the comparison observational. |
| Exposure separation | Whether people could encounter both variants and how repeat exposure or crossover was handled. | Do not assume independent comparison groups. |
| Observation scope | App, market, OS, eligible population, active dates, timezone and reporting cutoff. | Restrict the claim to the scope you can establish. |
| Concurrent changes | Changes to goals, budgets, destinations, app experience or asset availability during the comparison. | Preserve competing explanations for the result. |
| Measurement basis | Event definition, attribution source and window, metric denominator, and whether creative outcomes are directly attributed, modeled or allocated. | Resolve definitions before ranking business outcomes. |
| Maturity and precision | Comparable follow-up time, reporting completeness, outcome counts and uncertainty appropriate to the design. | Allow an inconclusive result; more impressions alone may not resolve it. |
| Analysis and stopping | Chosen outcome, planned review rule, treatment of repeated checks and an analysis that accounts for adaptive assignment if present. | Do not treat an ordinary significance calculation as sufficient validation. |
A clean match on market, OS and dates makes an observation easier to interpret. It cannot reconstruct unknown assignment or remove unobserved selection. Similarly, extending the observation period may improve precision while leaving the causal question unanswered.
Check the origin of creative-level outcomes explicitly. XMP’s automation and measurement discussion explains that some creative views can use weighted allocations when exact attribution is unavailable. That is a reason to inspect your report’s methodology, not evidence that your Moloco report uses that method. A displayed creative result is not automatically an independently measured outcome.

A practical review: two openings in one campaign
Illustrative scenario, with no performance results: a game team uploads two videos that differ only in their opening. Both run in an automatically optimized campaign. The team wants to decide which opening should guide the next production brief.
First, preserve the two files and confirm that the remaining footage, copy, CTA and destination match. Then identify the shared active period and whether either asset was paused, replaced or introduced later. Export only the reporting fields actually available; do not assume that assignment histories or user-level exposure records exist.
Next, ask the campaign owner whether the variants belonged to a documented experiment. If the available evidence is simply that both were uploaded and the system optimized delivery, record assignment as unverified. Even if one later shows a stronger reported result, the production brief should describe it as a candidate opening for further testing.
For a useful handoff, complete these five statements:
- Question: specify whether the team is choosing what to run now or estimating the effect of changing the opening.
- Observation: enter the actual metric, denominator, source, period, timezone and cohort age. Avoid an unqualified “winner” label.
- Evidence limit: state what is unknown about assignment, exposure, measurement or precision.
- Operating decision: record whether to retain, investigate, revise or stop, using your business constraints.
- Next learning step: name the evidence needed to strengthen the claim and who will obtain it.
If the team needs a causal answer, ask Moloco to confirm the available experimental design for that specific question, including assignment, interaction with optimization and reporting. The supplied public pages do not establish account eligibility or configuration steps.
Keep this conclusion attached to the asset version when it returns to production. XMP’s creative production workflow guide explains how asset organization and performance context support creative and media collaboration. That operational connection helps preserve learning; it does not itself validate an experimental claim.
Choose the next action in evidence order
- Validate measurement. Confirm the event, attribution basis and source of the creative-level result. Investigate missing or inconsistent data before interpreting creative quality.
- Check maturity and sufficiency. Compare equivalent outcome windows and assess uncertainty using a method suitable for the documented design. Leave sparse or immature evidence inconclusive.
- Connect cost with value. Review acquisition cost alongside the downstream action or revenue that matters to the app. A cheaper install alone cannot establish a better business outcome.
- Choose an action and label the claim. Make a conditional operating choice when that is all the evidence supports. If future production depends on isolating a creative effect, obtain a verified experiment design before promoting the observation into a causal conclusion.