Resources · 59
Retention cohorts: compare observable windows
Define entry, return and horizon, distinguish no return from an unobservable period and compare cohorts with completed observation windows.
· 4 min
What this guide helps achieve
- Define entry
- Name the return
- Respect observable time
- Check measurement
Quick check
- Does a blank cell mean zero?
- Can recent and older cohorts be compared?
- Does higher retention prove a product effect?
Step-by-step method
- 01
Define entry
Choose an entry event, timestamp and identity rule. Decide whether someone can enter repeatedly. A signup cohort answers a different question from a purchase cohort.
Deliverable: versioned inclusion rule.
- 02
Name the return
Specify the useful event, period and calculation mode. GA4 standard, rolling and cumulative modes use different definitions; their cells must not be compared as identical retention measures.
Deliverable: numerator and denominator definitions.
- 03
Respect observable time
At a given cutoff, a recent cohort has not reached every horizon. Mark those cells unobservable rather than zero. Compare the same horizon only when the relevant window has completed.
Deliverable: cell maturity mask.
- 04
Check measurement
Inspect tracking changes, timezone, identity merging and late events. GA4 documentation states that cohorts use device data without User-ID; do not describe them as certain cross-device person counts.
Deliverable: measurement limits and change log.
- 05
Compare in context
Show counts and acquisition periods alongside rates. Channels, promotions or audience mix may explain a difference. A descriptive comparison alone does not demonstrate the causal effect of a feature.
Deliverable: conclusion and explanations to investigate.
Four situations for testing the method
Fictional teaching cases. Each situation connects a signal, a check and a decision bounded by evidence; none represents a client outcome.
| Fictional situation | Concrete check | Decision and limit |
|---|---|---|
| Give cohorts the same observation time — An older cohort has had months to return, while a recent one is compared in the same column before reaching that age. | Define entry and return events, then compare cohorts at the same age with matching exclusions. Mark cells with incomplete observation periods. | Delay comparisons of immature periods. An apparent decline may reflect different observation time rather than weaker retention. |
| Separate observed CLV from a projection — A dashboard presents estimated future revenue for recently acquired customers as value already earned. | Separate observed contribution and projection, including horizon, selected costs, returns and retention assumptions. Compare cohorts at equal age. | Show alternative assumptions and a review date. Projected value is neither recognised revenue nor a guarantee of future retention. |
| Evaluate a prediction on a later period — A predictor uses a variable created after the event being predicted and appears exceptionally accurate on historical data. | Reconstruct information available at decision time and test a period held out from training. Compare cohorts and contexts. | Remove temporal leakage and measure business-relevant errors. Strong retrospective performance may not be reproducible in operation. |
| A real-time dashboard must expose its delay — The latest hour looks quiet while some events are still arriving after collection delays. | Show event time, receipt time and last refresh. Compare the same windows after closure and identify outstanding events. | Separate lower activity from provisional data. Screen refresh frequency does not establish collection completeness. |
Reusable worksheet
Complete with your authorised observations. These fields are a working template, not observed results.
| Field | Information to record |
|---|---|
| Inclusion | Event, identity and re-entry |
| Return | Event, mode and denominator |
| Calendar | Timezone, horizon and cutoff |
| Limits | Unobservable cell, collection and segments |
Worked example
Illustrative situation
Fictional example: last week’s cohort is compared with a cohort followed for two months.
Decision and expected evidence
The table masks future horizons and compares only the first observable period, with accompanying counts.
Distinguish the mechanisms
| Mechanism | Purpose | Check or limitation |
|---|---|---|
| Return in a period | Observe activity at that horizon | Do not confuse it with continuous return |
| Cumulative return | Observe any return so far | Do not read it as usage frequency |
| Unreached horizon | Identify an incomplete window | Do not record zero retention |
Management indicators
| Indicator | What it measures | First action |
|---|---|---|
| Fixed-horizon rate | Returns / members under the chosen rule | Publish counts and horizon |
| Mature cells | Fully observed windows | Exclude future horizons from comparison |
| Collection breaks | Periods with different instrumentation | Qualify affected trends |
Common pitfalls
- Do not confuse it with continuous return
- Do not read it as usage frequency
- Do not record zero retention
Frequently asked questions
Does a blank cell mean zero?
No. It can be unobservable, suppressed by a threshold or genuinely empty; name its state.
Can recent and older cohorts be compared?
Yes, at horizons observable in both, using common definitions.
Does higher retention prove a product effect?
No. Recruitment and context may change; causal conclusions require an appropriate design.
Official references
References consulted: . The method and worksheet propose checks to adapt to your context; they do not constitute certification.
