Resources · 90
Synthetic data: verify privacy and task utility
Evaluate generated records before sharing them or using them to train a model.
· 3 min
What this guide helps achieve
- Specify what must be preserved
- Map potential disclosure
- Separate development from evaluation
- Test disclosure scenarios
- Choose a sharing model
Quick check
- Does synthetic mean anonymous?
- Should rare cases be preserved?
- Is one score enough?
Step-by-step method
- 01
Specify what must be preserved
Choose a task: testing a form, comparing an algorithm or reproducing distributions. Record useful variables, relationships and rare cases. Tables that look similar do not establish task utility.
Output: Useful-variable and case specification.
- 02
Map potential disclosure
Record source data, generator access, direct identifiers and rare combinations. Replacing names often amounts to pseudonymisation. A synthetic label alone does not establish anonymisation.
Output: Source and risk map.
- 03
Separate development from evaluation
Keep an authorised real reference set outside generator tuning. Compare the task and subpopulations using a predefined method. Never copy real records into public examples.
Output: Independent versioned protocol.
- 04
Test disclosure scenarios
In an authorised environment, investigate near copies, rare values and linkage with other datasets. Record attacker assumptions and access. A negative test is not a universal guarantee.
Output: Scenario and limitation report.
- 05
Choose a sharing model
Compare public release, restricted access and protected environments. Approve recipients, limits and retention. Prepare withdrawal and reassessment when available sources or linkage techniques change.
Output: Sharing and reassessment decision.
Fictional acceptance-test scenarios
These proposed cases are not client observations. Adapt data, permissions and acceptance criteria to your authorised environment.
| Situation to exercise | Result to check | Evidence to retain |
|---|---|---|
| A rare combination appears in a synthetic dataset. | Examine possible linkage to accessible data; generated origin alone does not establish anonymity. | Linkage scenario, access used, outcomes and limitations. |
| Means are preserved but a small subgroup disappears. | Compare authorised use-case outcomes by subgroup and disclose lost utility. | Group definitions, counts and relevant differences. |
| A new dataset is generated from a changed source. | Repeat utility and privacy checks before release; an old assessment does not establish current safety. | Source version, test procedure and release decision. |
Reusable worksheet
Complete with your authorised observations. These fields are a working template, not observed results.
| Field | Information to record |
|---|---|
| Use | Task, recipients and necessary variables |
| Evaluation | Independent reference, segments and criteria |
| Disclosure | Authorised scenarios, outcomes and limits |
| Decision | Access, owner and review date |
Worked example
Illustrative situation
Fictional example: generated applicant records reproduce a rare combination of location and career history.
Decision and expected evidence
The team removes that case from release, tests remaining utility and selects restricted access instead of open publication.
Distinguish the mechanisms
| Mechanism | Purpose | Check or limitation |
|---|---|---|
| Masking | Remove obvious identifiers | Combinations may remain identifying |
| Synthetic generation | Create artificial records | Assess memorisation and utility |
| Protected access | Restrict users and outputs | Test exports too |
Management indicators
| Indicator | What it measures | First action |
|---|---|---|
| Near copies | Similarity to source records | Review affected cases |
| Segment utility | Performance on the defined task | Report underrepresented groups |
| Permitted outputs | Exports actually allowed | Test restrictions |
Common pitfalls
- Combinations may remain identifying
- Assess memorisation and utility
- Test exports too
Frequently asked questions
Does synthetic mean anonymous?
No. Assess context, techniques and reidentification possibilities; a marketing label is not evidence.
Should rare cases be preserved?
Only according to the task and risk. Document the trade-off and evaluate rare populations separately.
Is one score enough?
No. Utility, privacy and coverage answer different questions. Keep their results separate.
Official references
References consulted: . The method and worksheet propose checks to adapt to your context; they do not constitute certification.
Continue exploring
Related analysis
