Resources · 58
RAG: verify document access rights and isolation
Test retrieval, citations, caches and revocation so a document assistant respects each user’s permissions.
· 8 min
What this guide helps achieve
- Define boundaries
- Filter before exposure
- Separate caches
- Exercise revocation
Quick check
- Is a separate vector namespace enough?
- Should permissions be in the prompt?
- How can confidential documents be avoided in testing?
Step-by-step method
- 01
Define boundaries
Assign each document an owner, scope and access rule. OWASP describes cross-context leakage from misaligned vector-store permissions. Our proposed method also treats chunks and metadata as protected objects.
Deliverable: document, group and scope inventory.
- 02
Filter before exposure
Enforce authorisation in the retrieval service before a chunk reaches the model. Cover vector search, lexical search, graphs and direct downloads. A prompt instruction cannot replace server-side authorisation.
Deliverable: decision points and missing-identity behaviour.
- 03
Separate caches
Inventory retained search results, conversations, citations and answers. Check that reuse depends on the relevant access context. An index field naming a customer does not by itself prove isolation.
Deliverable: cache keys and separation scenarios.
- 04
Exercise revocation
Using non-sensitive test documents, remove access, delete a document and change a group. Replay a conversation opened before the change. Measure when chunks, links and retained answers become unavailable.
Deliverable: timestamped revocation evidence and gaps.
- 05
Keep negative tests
Ask the same question as an authorised user, another group and an unknown identity. Inspect filenames, excerpts, citations and logs. Replay after index or connector changes; a few passing cases only establish limited coverage.
Deliverable: test matrix and correction owner.
Four situations for testing the method
Fictional teaching cases. Each situation connects a signal, a check and a decision bounded by evidence; none represents a client outcome.
| Fictional situation | Concrete check | Decision and limit |
|---|---|---|
| Apply permissions before composing the answer — Two departments with different rights use an assistant that queries one corpus without consistent filtering. | Create test accounts and fictional documents with distinct rights. Check retrieval, model context, citations and refusals. | Block unauthorised access before generation and test human handoff. A prompt asking the model to respect rights is not access control. |
| Revoke retained RAG context as well — A document becomes inaccessible, but an answer cache keeps returning its contents to a user whose rights changed. | Replay a known question after revocation and check retrieval, caches, citations and authorised direct access. Measure the intended propagation delay. | Handle copies and secondary paths with revocation. Filtered primary retrieval does not establish that every access path is closed. |
| Hybrid retrieval must not bypass authorisation — Vector search filters a department’s documents, but lexical search or a graph relation still reveals a forbidden passage. | Replay identical questions and accounts across each retrieval route and their fusion. Trace identity and rights through cited passages. | Apply a consistent policy to every route before final ranking. Complementary engines do not replace access control. |
| A separate vector namespace does not prove isolation — Two customers have separate vector spaces, but citation resolution accepts a document identifier from the other customer. | Cross-test fictional identifiers, accounts, retrieval and citation resolution. Check service-side denial and absence of content in errors. | Verify authorisation for every object and access path. Storage separation is useful but does not alone establish end-to-end isolation. |
Reusable worksheet
Complete with your authorised observations. These fields are a working template, not observed results.
| Field | Information to record |
|---|---|
| Document / chunk | Owner and scope |
| Test identity | Allowed group, other group, missing identity |
| Access route | Retrieval, citation, file or cache |
| Result | Expected, observed, date and correction |
Worked example
Illustrative situation
Fictional example: a staff member changes teams but an earlier conversation still cites a withdrawn document.
Decision and expected evidence
The test follows retrieval and the conversation cache, then confirms refusal and absence of citations after correction.
Distinguish the mechanisms
| Mechanism | Purpose | Check or limitation |
|---|---|---|
| Retrieval filter | Limit retrieved chunks | Cover lexical routes and direct links too |
| Refusal prompt | Guide response behaviour | Do not delegate authorisation to it |
| Answer cache | Reuse a result | Check permission context before reuse |
Management indicators
| Indicator | What it measures | First action |
|---|---|---|
| Observed leaks | Test data exposed outside scope | Block affected use and inspect cause |
| Revocation delay | Time until all tested routes refuse | Name caches still accessible |
| Route coverage | Connectors and retrieval modes exercised | Identify untested paths |
Common pitfalls
- Cover lexical routes and direct links too
- Do not delegate authorisation to it
- Check permission context before reuse
Frequently asked questions
Is a separate vector namespace enough?
It helps partition data, but caches, file links and tools must honour the same boundary.
Should permissions be in the prompt?
Useful context may reach the model; the service exposing data must enforce access decisions.
How can confidential documents be avoided in testing?
Create fictional sentinel documents for each group with distinct markers, then inspect answers and citations.
RAG: access rights, citations and caches
An accurate answer can disclose a forbidden document. Retrieval relevance, citation validity and access authorisation are separate checks. The service providing passages must enforce the current identity’s rights before exposing them to the model.
Fictional CIE teaching set: four technical identities, six documents and three states. It illustrates a proposed policy without querying your system, authenticating real users or measuring product security. OWASP references describe risk families; the scenarios, rules and numbers below are our examples.
References consulted on .
Explore the fictional document policy
Choose identity, document, state and access path. The result is the decision expected under this policy, not an observed test of your application. All four paths should enforce the same access decision.
Local browser calculation with no connection to an AI model, vector database or account. No free-text field, secret or document to send.
Matrix of 24 expected decisions
Ops has group ops; Alpha has alpha; Beta has beta. The unauthenticated identity has no access. Documents without an explicit rule are denied.
10 allowed accesses across 24 fictional combinations
| Identity / document | Cross-team procedure | Alpha note | Beta note | Ops file | Old Alpha note | Unclassified document |
|---|---|---|---|---|---|---|
| Ops service | Allowed | Allowed | Allowed | Allowed | Allowed | Denied |
| Alpha service | Allowed | Allowed | Denied | Denied | Allowed | Denied |
| Beta service | Allowed | Denied | Allowed | Denied | Denied | Denied |
| Unauthenticated | Denied | Denied | Denied | Denied | Denied | Denied |
Three states with reproducible outcomes
| Fictional state | Policy change | Allowed accesses / 24 |
|---|---|---|
| 1 · Initial rights | Ops has group ops; Alpha has alpha; Beta has beta. The unauthenticated identity has no access. Documents without an explicit rule are denied. | 10 / 24 |
| 2 · Alpha group removed | Alpha remains authenticated but no longer has any required group. The cross-team procedure also requires one of ops, alpha or beta, so Alpha loses access to it as well. Ops and Beta groups are unchanged. | 7 / 24 |
| 3 · Group removed and old note deleted | Alpha’s group removal persists. The old Alpha note is deleted and denied to every identity, including Ops. Other document rules stay unchanged. | 6 / 24 |
These counts describe a small closed set, not a security rate. State changes are instant in this demonstration. In production, measure the actual delay on every path; revoking a right cannot retroactively erase a copy already read or exported.
Twelve replay cases with positive controls and expected evidence
Fictional scenarios. Buttons populate the demonstration and run no penetration test. To assess an application, compare each expectation with an actual trace from an authorised environment using sentinel data.
| Situation and reference | Demonstration input | Decision and evidence to retain |
|---|---|---|
| The group’s own document | Alpha service Alpha note 1 · Initial rights Vector retrieval | Allowed in this example Positive control: retain the permitted fictional passage and service decision. A system denying everything does not pass the functional test. |
| Another group’s document | Beta service Alpha note 1 · Initial rights Vector retrieval | Denied in this example Check that the passage, sensitive title and metadata are absent from data supplied to the model. |
| The secondary lexical branch | Beta service Alpha note 1 · Initial rights Lexical / hybrid retrieval | Denied in this example Retain each branch’s decisions before fusion and the final result. No vector leak does not establish lexical coverage. |
| A known citation identifier | Beta service Alpha note 1 · Initial rights Citation / file resolution | Denied in this example Call citation resolution using the Beta test identity; retain the object denial, not just the absence of a link in the answer. |
| An old cached answer | Alpha service Alpha note 2 · Alpha group removed Cached answer or passage | Denied in this example Create the answer before group removal and replay afterwards. Check that old passages are not returned under current rights. |
| The cross-team procedure after removal | Alpha service Cross-team procedure 2 · Alpha group removed Vector retrieval | Denied in this example This document requires an explicit group; remaining authenticated is insufficient. Retain the rule and membership change explaining the denial. |
| The unchanged group | Beta service Beta note 2 · Alpha group removed Lexical / hybrid retrieval | Allowed in this example Beta positive control: confirm that revocation handling does not block another authorised group. |
| The deleted but remembered document | Ops service Old Alpha note 3 · Group removed and old note deleted Cached answer or passage | Denied in this example Try an old citation and cache after deletion. Record object state and refusal to return it; separately verify removal of copies under your policy. |
| A document without a rule | Ops service Unclassified document 1 · Initial rights Vector retrieval | Denied in this example Even Ops has no implicit privilege. Retain the handling of an unclassified document and the process for correcting its access rule. |
| Access without an authenticated identity | Unauthenticated Cross-team procedure 1 · Initial rights Citation / file resolution | Denied in this example Check citation paths and their errors: reserved content must not be exposed during a denied attempt. |
| A source claims higher authority | Beta service Ops file 1 · Initial rights Cached answer or passage | Denied in this example A sentence inside the document changes neither groups nor service rules. Retain the technical denial even if model wording is persuasive. |
| The Ops file remains allowed | Ops service Ops file 3 · Group removed and old note deleted Citation / file resolution | Allowed in this example Positive control after both changes: establish this specific access without granting Ops the unclassified or deleted document. |
Document six technical boundaries
- Collection
- Retain document origin, version and status. Internal content can also carry hostile instructions. OWASP · LLM01:2025 Prompt Injection →
- Authorisation
- Check identity, object and groups in the service. Prompt text is not the permissions engine. OWASP · API1:2023 Broken Object Level Authorization →
- Generation and rendering
- Validate outputs for their destination: text, HTML, query or tool argument. A plausible answer is not directly executable input. OWASP · LLM05:2025 Improper Output Handling →
- Tool action
- Separate reading, proposing and modifying. Limit function, permission and autonomy to the task; read access does not authorise sending. OWASP · LLM06:2025 Excessive Agency →
- Configuration
- Keep secrets out of the system prompt. Confidentiality of its wording does not replace access controls. OWASP · LLM07:2025 System Prompt Leakage →
- Consumption
- Set request, duration and action limits, then exercise stopping and degraded operation with a demonstration budget. OWASP · LLM10:2025 Unbounded Consumption →
Useful traces without copying documents
Link test identifier, policy version, technical identity and effective groups, document identifier/version, access path, decision and timestamp. For revocation, add the change time and first observed denial per path. Use demonstration markers instead of secrets; restrict logs to necessary information and protect their access.
Continue with prompt injection testing →
Sources, editions and scope of the example
OWASP 2025 and API 2023 editions are explicitly identified. NIST AI 100-2 E2025 is a taxonomy published in March 2025; its page notes an April PDF correction and a June 2025 potential-update note. None certifies this simulator or your application.
- OWASP · LLM08:2025 Vector and Embedding Weaknesses · 2025
- OWASP · LLM01:2025 Prompt Injection · 2025
- OWASP · LLM06:2025 Excessive Agency · 2025
- OWASP · LLM05:2025 Improper Output Handling · 2025
- OWASP · LLM07:2025 System Prompt Leakage · 2025
- OWASP · LLM10:2025 Unbounded Consumption · 2025
- OWASP · API1:2023 Broken Object Level Authorization · 2023
- NIST AI 100-2 E2025 · Adversarial Machine Learning · Final · March 2025 ·
RAG relevance and ranking
Relevance and permission need separate checks
A relevant document may be forbidden to the requester. Build the evaluation corpus within accessible scope and test denials independently of ranking scores.
- Keep permissions and document versions with the test set.
- Check revocation, deletion and citation paths before measuring relevance.
- Do not use an nDCG improvement as evidence that permissions are respected.
Explore ranking measures and relevance judgments →
Stanford CS276 · Retrieval evaluation · 2019 course archive · PDF pp. 3, 5–6 · consulted 2026-10-23
NIST TREC · Reciprocal rank definition · Repository snapshot 35cf096ba6d5ae08ca4803c056b31ed61a56a6d4 · consulted 2026-10-23
Official references
References consulted: . The method and worksheet propose checks to adapt to your context; they do not constitute certification.
- OWASP — LLM08:2025, Vector and Embedding Weaknesses
- OWASP · LLM01:2025 Prompt Injection
- OWASP · LLM06:2025 Excessive Agency
- OWASP · LLM05:2025 Improper Output Handling
- OWASP · LLM07:2025 System Prompt Leakage
- OWASP · LLM10:2025 Unbounded Consumption
- OWASP · API1:2023 Broken Object Level Authorization
- NIST AI 100-2 E2025 · Adversarial Machine Learning
- Stanford CS276 · Retrieval evaluation
- NIST TREC · Reciprocal rank definition
