Using all visits as the denominator can make traffic changes resemble checkout improvement.
Commerce funnels with explicit denominators and confirmed outcomes
Define each step as an observable event. Separate a checkout attempt from an accepted order and confirmed payment.
Research context: Microsoft Research · Reliable online experiments ↗
Choose a denominator fitting the change, such as eligible product visits. Deliberately deduplicate shoppers, orders and payment events.
Try the pattern
Task and expected result
- 01
Write the primary metric with numerator and denominator.
- 02
Explain exclusion of test orders and duplicate webhooks.
- 03
Create separate definitions for paid and credit orders.
How to investigate value
Define audience, exposure, denominator and primary outcome before starting. Record what could invalidate the data.
Watch for side effects
Data loss, incorrect variant allocation, returns/refunds, margin, accessibility and mobile performance.
This is a proposed investigation, not a measured result. Choose one hypothesis and define the audience before testing.
Build a measurement plan for this concept ↗