Resources · 59

Retention cohorts: compare observable windows

Define entry, return and horizon, distinguish no return from an unobservable period and compare cohorts with completed observation windows.

· 4 min

Empty intelligence archive with dossiers and maps AI-generated editorial illustration

What this guide helps achieve

  • Define entry
  • Name the return
  • Respect observable time
  • Check measurement

Quick check

  • Does a blank cell mean zero?
  • Can recent and older cohorts be compared?
  • Does higher retention prove a product effect?

Step-by-step method

  1. 01

    Define entry

    Choose an entry event, timestamp and identity rule. Decide whether someone can enter repeatedly. A signup cohort answers a different question from a purchase cohort.

    Deliverable: versioned inclusion rule.

  2. 02

    Name the return

    Specify the useful event, period and calculation mode. GA4 standard, rolling and cumulative modes use different definitions; their cells must not be compared as identical retention measures.

    Deliverable: numerator and denominator definitions.

  3. 03

    Respect observable time

    At a given cutoff, a recent cohort has not reached every horizon. Mark those cells unobservable rather than zero. Compare the same horizon only when the relevant window has completed.

    Deliverable: cell maturity mask.

  4. 04

    Check measurement

    Inspect tracking changes, timezone, identity merging and late events. GA4 documentation states that cohorts use device data without User-ID; do not describe them as certain cross-device person counts.

    Deliverable: measurement limits and change log.

  5. 05

    Compare in context

    Show counts and acquisition periods alongside rates. Channels, promotions or audience mix may explain a difference. A descriptive comparison alone does not demonstrate the causal effect of a feature.

    Deliverable: conclusion and explanations to investigate.

Four situations for testing the method

Fictional teaching cases. Each situation connects a signal, a check and a decision bounded by evidence; none represents a client outcome.

Fictional situationConcrete checkDecision and limit
Give cohorts the same observation time — An older cohort has had months to return, while a recent one is compared in the same column before reaching that age.Define entry and return events, then compare cohorts at the same age with matching exclusions. Mark cells with incomplete observation periods.Delay comparisons of immature periods. An apparent decline may reflect different observation time rather than weaker retention.
Separate observed CLV from a projection — A dashboard presents estimated future revenue for recently acquired customers as value already earned.Separate observed contribution and projection, including horizon, selected costs, returns and retention assumptions. Compare cohorts at equal age.Show alternative assumptions and a review date. Projected value is neither recognised revenue nor a guarantee of future retention.
Evaluate a prediction on a later period — A predictor uses a variable created after the event being predicted and appears exceptionally accurate on historical data.Reconstruct information available at decision time and test a period held out from training. Compare cohorts and contexts.Remove temporal leakage and measure business-relevant errors. Strong retrospective performance may not be reproducible in operation.
A real-time dashboard must expose its delay — The latest hour looks quiet while some events are still arriving after collection delays.Show event time, receipt time and last refresh. Compare the same windows after closure and identify outstanding events.Separate lower activity from provisional data. Screen refresh frequency does not establish collection completeness.

Reusable worksheet

Complete with your authorised observations. These fields are a working template, not observed results.

FieldInformation to record
InclusionEvent, identity and re-entry
ReturnEvent, mode and denominator
CalendarTimezone, horizon and cutoff
LimitsUnobservable cell, collection and segments

Worked example

Illustrative situation

Fictional example: last week’s cohort is compared with a cohort followed for two months.

Decision and expected evidence

The table masks future horizons and compares only the first observable period, with accompanying counts.

Distinguish the mechanisms

MechanismPurposeCheck or limitation
Return in a periodObserve activity at that horizonDo not confuse it with continuous return
Cumulative returnObserve any return so farDo not read it as usage frequency
Unreached horizonIdentify an incomplete windowDo not record zero retention

Management indicators

IndicatorWhat it measuresFirst action
Fixed-horizon rateReturns / members under the chosen rulePublish counts and horizon
Mature cellsFully observed windowsExclude future horizons from comparison
Collection breaksPeriods with different instrumentationQualify affected trends

Common pitfalls

  • Do not confuse it with continuous return
  • Do not read it as usage frequency
  • Do not record zero retention

Frequently asked questions

Does a blank cell mean zero?

No. It can be unobservable, suppressed by a threshold or genuinely empty; name its state.

Can recent and older cohorts be compared?

Yes, at horizons observable in both, using common definitions.

Does higher retention prove a product effect?

No. Recruitment and context may change; causal conclusions require an appropriate design.

Official references

References consulted: . The method and worksheet propose checks to adapt to your context; they do not constitute certification.