Costs
Part of Reading campaign measurement with a sceptical eye
Zoopla and Atlassian campaign measurement case studies, checked for rigor
Campaign measurement case studies should separate vendor claims from methods, compare attribution and lift, inspect populations, and require local validation.
What to take away
- The Zoopla full funnel study is unverifiable from what is published. Treat its numbers as leads, not results.
- The Atlassian CTV story is partly verifiable, because Kantar is named as the brand lift provider.
- Neither account is independent proof that a channel, tool or agency works.
- Copy the measurement architecture and leave the headline percentage behind.
- Ask for the control group, the attribution window and any mid-campaign change before you reuse a claim.
Verdict on each case study
Short answer: one case fails the audit on published evidence, the other passes in part.
There is no hidden numerical trick here. The Zoopla study is unverifiable, because Google publishes it as a customer story with outcomes but no method note, no control group and no interval.
The Atlassian story is partial. Kantar ran the brand lift work, while LinkedIn measured reach and engagement inside its own platform. One layer can be checked in principle, and the other two cannot.
A sceptical reading of campaign measurement starts with two questions. Who published the number, and who paid for the study?
Case one: what Zoopla reports and what is missing
Think with Google describes a Zoopla study covering ad recall, search activity and conversions, plus a change to the audience made during the campaign. Those are the claim categories the public page supports.
The same page gives no control group, no attribution window and no confidence interval. Where the study quotes a percentage, this review records it as unverified rather than repeating it.
Case one: Zoopla
| Reported element | Useful question | Evidence still needed |
|---|---|---|
| Ad recall lift | Which instrument and which control group? | Questionnaire wording, sample, assignment, interval |
| Search activity | Branded or non branded queries? | Query source, baseline window, seasonality controls |
| Conversions | Attributed by which model? | Attribution window, deduplication, incrementality test |
| Audience change mid campaign | Was the rule set in advance? | Rule, timing, effect on the control group |
Zoopla study audit questions
- Primary vs secondary funnel outcomes
- Audience refinement comparability
- Lift vs attributed claims
- Optimization protection
- Dated analysis plan needed
Shorter versions of the same pattern, stripped of vendor framing, are collected in campaign measurement examples you can compare against this one.
Case two: what the Atlassian CTV story reports
LinkedIn's Atlassian CTV customer story presents three measurement layers. Kantar ran a brand lift study on brand attitudes, and LinkedIn measured incremental reach inside its own platform. The engagement comparison comes from exposure, not randomisation.
Each layer uses a different population and a different method. Blending them into one result inflates the claim, and the vendor account cannot carry causal weight for every line.
Case two: Atlassian CTV
| Measurement layer | Reported source | Transfer check |
|---|---|---|
| Brand attitudes | Kantar lift study | Sample, assignment, instrument, weighting, interval |
| Incremental reach | LinkedIn internal measurement | Deduplication scope and platform boundary |
| Lower funnel engagement | Exposure comparison | Selection, sequence, denominator, causality |
| Business decision | Internal interpretation | Alternative evidence and mature outcome |
Atlassian CTV measurement layers
Brand attitudes
- Reported source
- Kantar lift study
- Transfer check
- Sample, assignment, weighting
- Causality
- Experimental if designed
Incremental reach
- Reported source
- LinkedIn internal
- Transfer check
- Dedup scope, platform boundary
- Causality
- Platform-bounded
Lower-funnel engagement
- Reported source
- Exposure comparison
- Transfer check
- Selection, sequence, denominator
- Causality
- Not causal proof
Verdict: partial. The brand lift layer becomes checkable if Kantar publishes its design. The reach and engagement layers stay internal.
A five step audit before you reuse a claim
A five step audit
- Write down the publisher and the commercial relationship in one line.
- Map every number to one method and one population. Refuse to blend the layers.
- Demand the baseline, the comparison, the period, the denominator and the uncertainty.
- Record every campaign change made while the study was running.
- Reproduce one material claim on your own data before you copy the tactic.
That separation of layers is the core of campaign measurement best practices, which is worth reading before you sign a measurement contract.
Example: a bounded local replication
Example: a team wants to test the Atlassian CTV pattern on its own budget. Run four weeks, one channel, one primary outcome, and write the rules first.
- Holdoutkeep 10 to 20 percent of the target audience unexposed. That is a common design choice, not a universal rule.
- Instrumentcommission a third party for the brand survey, so the vendor does not grade its own work. Randomised assignment is the point of A/B testing.
- Decision ruleset the minimum effect you will act on before the test starts. Intervals matter more than point estimates, as the note on statistical significance explains.
- Budgeta four week CTV holdout test with a survey commonly lands in the low tens of thousands of dollars in the United States. Treat that as a planning range.
If the test clears your written threshold, the vendor claim turns into a local finding you can defend at budget review.
Where the rules bite
The FTC advertising substantiation policy requires a reasonable basis for objective claims before publication. That applies to your own performance claims as much as to a vendor's.
A holdout that withholds an ad is rarely a privacy problem. The survey data collected alongside it is, and Canadian teams handle that consent under PIPEDA.
Run the numbers against the campaign measurement checklist so nothing is forgotten once the results land.
