RAG on enterprise data: how to keep document permissions intact

The most common failure in enterprise GenAI isn’t a hallucination. It’s an assistant that answers a question correctly, using a document the person asking was never allowed to see.

It happens because of how most RAG pipelines are built. Documents are pulled from SharePoint, Confluence, file shares or a CRM; split into chunks; embedded; and written to a vector index. The access-control list that protected the original document is left behind at the first step.

From then on, retrieval only knows that a chunk is relevant. It has no idea who’s allowed to read it.

The rule: permissions travel with every chunk

The fix is a data engineering pattern, not a model setting:

1. Capture access metadata at ingestion. For every document, record who can read it (users, groups, roles) from the source system’s ACLs.
2. Copy it onto every chunk. Each chunk in the index carries the same access metadata as its parent document.
3. Filter during retrieval. When a user asks a question, resolve their identity and group memberships, and restrict the search to chunks they’re allowed to read.
4. Keep it in sync. Re-sync permissions on a schedule, and propagate revocations quickly.

Done this way, restricted content never reaches the model’s context, so it can’t appear in an answer.

Why filtering after retrieval isn’t enough

A tempting shortcut is to retrieve the top results first and drop the ones the user can’t see. Two problems:

• If most top results are restricted, too few remain to answer well, and quality drops for exactly the people with the least access.
• Restricted content may already have been logged, cached or passed to a reranker before being dropped.

Filter as part of the search, not after it.

Structured data needs the same treatment

RAG and agents increasingly query tables as well as documents. Here the same principle applies: the agent should query as the user, with row- and column-level security enforced by the platform (Unity Catalog on Databricks, row access and masking policies on Snowflake), not with a service account that can see everything.

Freshness is part of correctness

An assistant that quotes last year’s policy is wrong, even if it’s permitted. Treat document ingestion like any other production pipeline:

• deduplicate versions so only the current document is retrievable;
• set a freshness target per source;
• alert when a source stops updating.

Measure it

Build a small labelled test set: questions, the documents that should answer them, and users who should and shouldn’t get an answer. Run it on every change to chunking, embeddings or permissions logic. Permission tests should be pass/fail, not a score.

A checklist for your own assistant

• Are source ACLs captured at ingestion?
• Does every chunk carry its parent document’s access metadata?
• Is filtering applied during retrieval, using the user’s real identity?
• How fast does a revoked permission reach the index?
• Are old versions of documents removed?
• Is there an automated test that a restricted user gets no answer?

If any answer is “no” or “not sure”, the assistant isn’t ready for sensitive data yet. Our AI-ready data work starts with exactly these controls.