Resources · 46
Data reliability: diagnose a dashboard before making a decision
Separate a real decline from late data, broken collection and a changed definition.
· 4 min
What this guide helps achieve
- Connect a number to sources and transformations
- Detect incomplete periods
- Distinguish missing data from zero
- Repair without counting twice
Quick check
- When was the latest event processed?
- Is the displayed period closed?
- Has the denominator changed?
- Is a missing value converted to zero?
- Does replay create duplicates?
Step-by-step method
- 01
Document the contract
Write definition, unit, dimensions, scope, time zone, frequency, source and metric owner. Specify null handling, late corrections and changes to collection or consent. An identical formula can produce incomparable figures for different populations.
Deliverable: data contract and definition history.
- 02
Trace transformations
Connect source event, storage, filtering, deduplication, aggregation and chart. Record versions, currency conversions and join rules. Follow authorised sample records end to end without copying sensitive data into the diagnostic dossier.
Deliverable: lineage and reconciliation sample.
- 03
Measure three distinct dimensions
Track freshness, completeness and correctness separately. A completed job can omit a partition; expected volume can contain incorrect values. Set thresholds around the decision and tolerable delays instead of adopting a universal percentage.
Deliverable: checks, thresholds and measurement scope.
- 04
Qualify an anomaly
Compare source and output in the same window; inspect delays, missing events, filters, joins, collection and definition. Segment only where volume remains interpretable. Label partial data instead of presenting a decline as established fact.
Deliverable: diagnosis separating facts, hypotheses and unknowns.
- 05
Repair and replay
Fix the cause in a limited scope. Replay with stable identity, deduplication and total reconciliation; confirm propagation into exports and charts. Retain the before/after values needed for explanation without unnecessary individual detail.
Deliverable: reconciliation and correction evidence.
- 06
Expose quality status
Show last update, covered period and known limitations. Give every alert an owner and an action. Retest after source or schema changes; an overall average can hide a missing segment.
Deliverable: quality status and escalation procedure.
Proposed acceptance scenarios
Fictional scenarios to adapt in an authorised environment. They describe no client result or certification.
| Fictional situation | Suggested check | Evidence to retain |
|---|---|---|
| Two source records join to several states under the same key. | Compare resulting pairs, distinct identifiers and totals at the expected grain. A plausible sum does not establish the correct population. | Both source versions, join condition, before-and-after counts, exclusions and independent reference. |
| A batch arrives late and is replayed after correction. | Separate event time from processing time, check duplicate prevention and expose remaining gaps. Recent collection does not establish complete coverage. | Batch identifier, observation window, replay rule and source-to-output reconciliation. |
Worked example
Illustrative situation
Illustrative situation: a chart shows fewer orders while processing for one region is delayed.
Decision and expected evidence
The page labels the partial period, processing resumes without duplicates and totals are reconciled before any commercial conclusion.
Distinguish the mechanisms
| Mechanism | Purpose | Check or limitation |
|---|---|---|
| Freshness | Is data recent enough? | Latest processed event and delay |
| Completeness | Are expected items present? | Reconciled partitions and counts |
| Correctness | Do values follow the rule? | Expected results on known cases |
Management indicators
| Indicator | What it measures | First action |
|---|---|---|
| Delay | Time between event and availability | Inspect pipeline and queues |
| Coverage | Received items against expected items | Locate missing partitions |
| Reconciliation gaps | Differences between source and output | Find filters, joins or duplicates |
Common pitfalls
- Treating missing data as zero
- Confusing successful execution with correct output
- Changing definitions without annotation
- Replaying without deduplication
Frequently asked questions
Does a statistical alert prove an error?
No. It flags a difference for investigation. A real business change and a technical failure can create similar curves.
Which freshness threshold should be used?
One the decision can tolerate, with documented scope and method. Operational monitoring and monthly reporting have different needs.
Can partial data be shown?
Yes, if scope and partial status are clearly visible and users can distinguish an estimate from a finalised period.
What evidence makes a verification useful?
Keep source versions, grain, filters, keys, time window and row-level reconciliation. Check duplicates, missing data and freshness separately; an unchanged mean or total can conceal a different population.
Timelines: UTC and time zones
A revised total needs its dated snapshot
Delay, redelivery and an identifier conflict have different effects. Comparing dashboards requires preserving the window, cutoff and records actually included.
- Normalize instants before sorting or calculating elapsed time.
- Keep event time and observation time separately.
- Explain snapshot changes without treating late receipt as a new event.
Read the 12 cases and three dated snapshots →
Sources reviewed on · IETF · RFC 3339 · Python · PEP 495 · Python 3.12 · zoneinfo · IANA · Time Zone Database 2026e · IANA · 2026e Europe rules · OpenTelemetry · Logs Data Model
Syslog codes and collection evidence
Normalisation must preserve event meaning
In a distributed API, a PRI integer combines two codes. An opaque conversion may lose the message family or reverse severity. Test collection with identified events before inferring operational reliability.
- Keep facility, severity and raw value in separate fields.
- Distinguish source time, reception time and search availability.
- Expose rejected events, truncation, loss and conversion rules.
Read syslog codes and try the PRI calculation →
IANA · syslog Parameters · Registry updated 2016-04-26 · consulted 2026-10-23
ASD ACSC and partners · Best practices for event logging and threat detection · Published and reviewed 2024-08-22 · consulted 2026-10-23
Join cardinality, NULL and aggregates
Reconcile rows before interpreting a dashboard
A metric may change with identical source events: a join creates pairs, a filter removes rows and null measurements reduce contributions. Keep a reference at the source grain and inspect retained identifiers.
- Compare COUNT(*), distinct identifiers and non-null measurements.
- Inspect multiple matches before summing.
- Test the active predicate in ON and WHERE on an unmatched case.
Inspect pairs and aggregates in the workshop →
PostgreSQL 18 · Table expressions · PostgreSQL 18 · documentation reviewed 2026-10-23
PostgreSQL 18 · Aggregate functions · PostgreSQL 18 · documentation reviewed 2026-10-23
PostgreSQL 18 · Comparison functions · PostgreSQL 18 · documentation reviewed 2026-10-23
Official references
References consulted: . The method and worksheet propose checks to adapt to your context; they do not constitute certification.
- Google SRE — Data Processing Pipelines
- Google SRE — Implementing SLOs
- PostgreSQL 18 · Table expressions and joins
- IETF · RFC 3339
- Python · PEP 495
- Python 3.12 · zoneinfo
- IANA · Time Zone Database 2026e
- IANA · 2026e Europe rules
- OpenTelemetry · Logs Data Model
- PostgreSQL 18 · Aggregate functions
- PostgreSQL 18 · Comparison functions
- IANA · syslog Parameters
- IETF · R. Gerhards · RFC 5424 · The Syslog Protocol
- ASD ACSC and partners · Best practices for event logging and threat detection
