Careers
Part of Reading campaign measurement with a sceptical eye
Why campaign measurement projects go wrong, and how to prove a fix
Campaign measurement mistakes include vague scope, attribution overclaims, weak baselines, changed tests, immature value, quality gaps, and silent corrections.
What to take away
Fix the evidence chain, not only the presentation.
- A measurement label cannot repair a weak comparison.
- Changing settings or primary outcomes after seeing results changes the claim.
- Corrections should reach every person and decision affected by the defect.
Campaign measurement mistakes usually enter before the chart: an unclear decision, moving campaign boundary, incompatible baseline, or outcome that has not matured.
Diagnose the common failures
| Mistake | Consequence | Repair |
|---|---|---|
| Undefined campaign | Unrelated tactics enter the result | Freeze scope and version |
| Attribution called lift | Assigned credit is mistaken for effect | Separate claims and evidence |
| Weak before-and-after baseline | Seasonality or events explain change | Use a stronger comparison or limit claim |
| Midtest primary change | Result follows observed data | Preserve original and label exploration |
| Immature value | Recent cohorts look better or worse | Use fixed-age closes |
| Missing quality status | Incomplete data appear final | Gate and annotate reporting |
| Silent correction | Past decisions rest on wrong values | Issue a dated impact notice |
Campaign Manager 360 documents how attribution-model differences can create discrepancies and describes product-specific compatibility and date effects. The page supports checking settings before comparing reports, not treating any model as proof of causation. The gap between assigned credit and measured effect is the subject of marketing attribution.
Campaign Measurement Failures and Repairs
Mistake
- Undefined campaign
- Unrelated tactics enter result
- Attribution called lift
- Credit mistaken for effect
- Weak baseline
- Seasonality explains change
- Midtest primary change
- Result follows observed data
- Immature value
- Recent cohorts look better
- Missing quality status
- Incomplete data appear final
- Silent correction
- Decisions rest on wrong values
Consequence
- Undefined campaign
- Freeze scope and version
- Attribution called lift
- Separate claims and evidence
- Weak baseline
- Use stronger comparison
- Midtest primary change
- Preserve original, label exploration
- Immature value
- Use fixed-age closes
- Missing quality status
- Gate and annotate reporting
- Silent correction
- Issue dated impact notice
Repair
- Undefined campaign
- Attribution called lift
- Weak baseline
- Midtest primary change
- Immature value
- Missing quality status
- Silent correction
Pre-register the lift study and protect the design
Google's guide to setting up a Conversion Lift study describes eligibility, conversion actions, power, budget, and duration considerations within Google Ads. Meeting those product requirements does not remove broader concerns such as spillover, value maturity, or business cost.
Before launch, freeze these items in writing:
- Primary outcome and how it is measured.
- Minimum detectable effect and power.
- Planned duration and the read date.
- Exclusion rules for bot traffic, internal users, and contaminated geos.
Protect the Planned Lift Study
- Preserve original brief, settings, analysis plan
- Log delivery or eligibility changes by timestamp
- Separate operational monitoring from primary inference
- Show failed quality or power conditions
- Reconcile outcome and cost sources
- Trace every material correction to reports
When a contradiction appears, stop distribution of the affected claim. Recheck the source, population, period, formula, settings, and evidence design. Correct the wording and numbers, state what changed, and reassess decisions.
Prove the correction
A mistake is not closed when a dashboard changes. Trace the correction from the source record through processing, reporting, exports, and the decision owner. Preserve the prior value, reason, date, reviewer, and affected actions.
Worked example (numbers are typical ranges, not one client). Defect: a primary outcome changed midtest from a 30-day fixed-age close to a 7-day close. Reported lift was 5 to 7 percent. Corrected number using the original 30-day window was 2 to 3 percent, a typical shrink of 2 to 4 percentage points for immature windows.
Decision changed: the team cancelled a planned double-digit budget increase and kept the original spend.
Materiality test. Withdraw or quarantine the report if the defect changes the direction of the result, crosses a decision threshold, or alters a budget or reallocation decision. Use a dated footnote if the correction is smaller than the metric's typical noise and no decision moves.
Dated impact notice template:
- Prior value
- [old figure]
- New value
- [corrected figure]
- Reason
- [defect and source]
- Reviewer
- [name or role]
- Affected decisions
- [list]
Then test whether the same defect can recur, one of the common campaign measurement questions worth answering in writing. This turns an error list into an operating control.
Such claims depend on usable source data. The GAO data reliability guide describes assessing data in relation to the intended use and documenting the work. That public-sector guide does not certify a local dataset. It supports a written test of completeness, accuracy, and known limits before release.
The FTC advertising substantiation policy says objective advertising claims require a reasonable basis before dissemination. Use that U.S. policy to challenge public performance statements about campaign results. It does not define the right method for every claim or replace advice on the facts involved.
Keep the evidence record beside the decision so a reviewer can reproduce the reasoning. Set the next review date and the change that would trigger an earlier check. The same failures one level up, at metric rather than campaign level, are listed in marketing KPIs mistakes.
Common questions
Is last-click reporting a mistake?
Not when it is clearly defined and suitable for an operating task. The mistake is presenting its assigned credit as incremental effect.
Can a failed test still teach something?
Yes. Delivery, recruitment, contamination, data quality, or power failures can improve the next design, but they do not support the intended effect claim.
When should a campaign report be withdrawn?
Withdraw or clearly quarantine it when a material defect could change interpretation or action, then publish a dated correction.
