2026-02-22 · Eunji Park

Instrumentation checklists for holdout purists

Holdouts break for boring reasons: late-arriving traits, duplicated accounts after a merge, or a campaign firing outside the experiment window. Our checklist walks engineers and marketers through the same sequence so fewer surprises surface on day ten.

We emphasize naming conventions for experiment keys and downstream dashboards. Consistency sounds trivial until three teams each invent their own prefix and finance cannot reconcile results.

The article closes with a reconciliation ritual: compare exposed counts to billing reality weekly, not only at readout. Small drift caught early saves reputations when leadership asks uncomfortable questions.