Securing Enterprise RAG: Access Control, Multi-Tenancy, and Vector Store Security

Every enterprise RAG deployment quietly performs the same risky move: it copies documents out of systems that enforce permissions — SharePoint, Confluence, Google Drive, data lakes — into a vector index that, by default, enforces none. The retriever will hand any indexed chunk to any caller. Securing RAG means putting authorization back at the only place it can hold: retrieval time.

This guide covers the three layers that decision, taken together, actually requires: document-level access control enforced at query time, tenant isolation for multi-tenant B2B products, and the security posture of the vector store itself — including the least intuitive part, that the embeddings are themselves recoverable, sensitive data.

1. By the numbers

2. Why RAG breaks your existing access model

Source systems enforce permissions at read time: when a user opens a file, the system checks their identity against an ACL. A RAG pipeline reads those files once, with a privileged service account, then serves their contents forever through a different door. The original ACL check never fires again. Unless permissions travel with the content and get re-evaluated on every query, your RAG system is a permission-laundering machine: it takes documents readable by few and makes their contents retrievable by many.

The failure is worse than classic over-broad search, because the LLM synthesizes. A user who could never open the compensation file can still ask "what's the salary band for L7 engineers?" and get a fluent answer grounded in a chunk they were never entitled to see. Post-hoc output filtering cannot reliably catch this — the leaked fact arrives paraphrased. The only robust control point is before the model sees the chunk: filter the retrieval, not the response.

Platform vendors have converged on exactly this design. Amazon Bedrock's managed Knowledge Bases ship connectors for enterprise repositories "along with document-level permission filtering using Access Control Lists (except for Web Crawler) at retrieval time"^[4] — and AWS notes that document-level permissions are a managed-offering capability, not something you get for free when you roll your own pipeline on a raw vector store.^[4] Azure AI Search enforces the same idea through query-time checks: attach the caller's Microsoft Entra token to the query via the x-ms-query-source-authorization header, and the service compares the token's user, group, and scope claims against permission metadata stored alongside each indexed document, returning only documents the caller is authorized to read.^[2]

3. Document-level access control: three patterns, one sharp edge

The metadata-filter pattern deserves respect and suspicion in equal measure. Done well — Azure's documented approach uses a filterable, non-retrievable field of group identifiers and a search.in filter function that keeps response times subsecond even with long group lists^[5] — it is fast, platform-agnostic, and generally available. But Microsoft's own description is candid: "There's no authentication or authorization through the security principal. The principal is just a string, used in a filter expression."^[5] If your application forgets the filter on one query path, nothing downstream notices. Token-based enforcement moves that failure mode from your code into the platform, which is why it's worth adopting as it matures.

The sharp edge nobody budgets for is ACL synchronization. Query-time enforcement evaluates the caller against permission metadata already stored in the index — so a permission revoked in SharePoint or your IdP is still honored in RAG results until the next sync writes the change through. Azure's docs state this plainly: permission changes in the source system "are only reflected in search results after that metadata is synchronized to the index," via an indexer run, push-API update, or policy refresh.^[2] That lag is a standing access-revocation gap. Treat ACL freshness as an SLA with a number attached, and design your connector layer to honor it — the connector-side mechanics are covered in depth in /guides/enterprise-rag-connectors.

4. Multi-tenant isolation: pick your blast radius

For B2B SaaS teams putting RAG into their product, the access-control question compounds: it's no longer which employee sees which document, but which *customer's* corpus a query can touch at all. A cross-tenant leak is not an incident, it's a churn event and probably a breach notification. The architecture decision is which isolation boundary you buy: a shared index with tenant filters, an index per tenant, or a dedicated deployment per tenant.

The honest way to choose is by blast radius, not by cost sheet. A shared index with tenant tagging fails open: one bug in filter construction — a null tenant ID, an OR where an AND belonged, a new query path that skips the middleware — and tenant A retrieves tenant B's contracts. An index or namespace per tenant fails closed: a routing bug sends a query to the wrong index and returns wrong-but-isolated results, or nothing. Microsoft's guidance also documents a hybrid: dedicated services for the largest or most regulated tenants, index-per-tenant for the long tail.^[6] That maps cleanly to how most B2B contracts actually tier.

5. The vector store is a database — secure it like one

Vector stores tend to enter organizations through ML teams, not database teams, and they often skip the hardening checklist every relational store gets by default. The baseline is well documented by the platform vendors. Azure AI Search encrypts all data at rest with 256-bit AES using FIPS 140-2 compliant, service-managed encryption, enforces TLS 1.2 or 1.3 on every connection, and supports customer-managed keys as a second encryption layer plus Private Link endpoints for network isolation.^[7] Amazon OpenSearch Service goes further and makes hygiene a precondition: enabling fine-grained access control *requires* HTTPS for all traffic, encryption of data at rest, and node-to-node encryption — and once enabled, it cannot be disabled.^[3] On Google Cloud, Vertex AI's generative-AI security controls include customer-managed encryption keys, Access Transparency, and Data Access audit logs.^[8]

Fine-grained access control in OpenSearch is also a working model of what "least privilege inside the index" looks like for RAG: roles can combine cluster, index, document-level, and field-level permissions, where document-level security restricts which documents a role can see via a query, and field-level security or field masking hides or anonymizes specific fields in results.^[3] That is the same enforcement primitive as RAG permission filtering, applied one layer down — and it composes with it.

Audit logging is where assumptions go to die. Compliance teams routinely assume the search layer can answer "who retrieved this document?" — and it often can't. Azure AI Search, for instance, logs create, read, update, and delete operations and query text, but "doesn't log user identities, so you can't refer to logs for information about a specific user."^[7] If per-user retrieval attribution matters for your audit posture — and for HR, legal, or clinical corpora it will — you must log it yourself at the application or gateway tier, where the caller's identity is known. Verify this capability during vendor selection, not during your first incident.

6. Embeddings leak: the inversion problem

The most common security shortcut in RAG architectures is treating vectors as safely abstract — "we only replicate the embeddings, not the text." The research says otherwise. Morris et al. framed embedding inversion as a controlled generation problem and found that "a multi-step method that iteratively corrects and re-embeds text is able to recover 92% of 32-token text inputs exactly," and demonstrated recovery of full names from a dataset of clinical notes.^[1]

The architectural consequence is simple: every control you apply to source documents applies to their embeddings. Vector stores inherit the data classification of the most sensitive document they index. Embeddings cross the same residency boundaries as text for sovereignty purposes. And an attacker with read access to the raw vectors — a leaked API key, an over-permissive analytics replica, a decommissioned index that never got deleted — should be modeled as having read access to the corpus.

7. Honest objections

"Metadata filtering is just a WHERE clause — one bug and it all leaks, so it's disqualified." Partly right, and the blast-radius framing above takes it seriously. But the steelman cuts the other way too: filter-based trimming is the pattern the platform vendors themselves document and ship,^[5] and per-tenant physical isolation has its own failure modes — tenant-routing bugs, index-count ceilings,^[6] and an operational surface that grows with every customer. The defensible position is not "filters are unsafe" but "filters assembled ad hoc in application code are unsafe." Centralize filter construction in one enforced middleware path, prefer platform-enforced token checks where available, and test the negative case in CI.

"The model is the real leak, not retrieval." Also partly right. Retrieval-time access control is necessary, not sufficient: a RAG-backed agent with tool access can still be steered into exfiltrating whatever it can legitimately retrieve, which is a governance problem — identity for agents, egress controls, human checkpoints — covered in /guides/agent-governance-guide. But the dependency runs one way. If retrieval is unfiltered, no amount of model-side guardrailing saves you; if retrieval is correctly scoped, model-side failures are bounded by what the caller was allowed to see anyway. Fix retrieval first.

"Inversion is a lab attack; nobody is reconstructing our corpus from vectors." Fair as a likelihood estimate, weak as an architecture principle. The inversion result matters less as an active threat than as a classification ruling: it removes the argument that a vector store is out of scope for data-protection controls because it holds "derived" data.^[1] The cost of accepting that ruling is low — encrypt, isolate, delete on offboarding — and the cost of arguing it in front of a regulator after a vector-store exposure is not.

8. The read

Securing enterprise RAG is three decisions, made explicitly rather than inherited from defaults. First, enforcement point: document-level permissions are enforced at query time, in the retrieval platform where possible, in one centralized middleware where not — never post-hoc on model output. Second, tenancy: choose isolation by blast radius and contract tier — shared-index filtering for the cost-sensitive long tail, index- or service-per-tenant where a filter bug would be unsurvivable. Third, classification: the vector store carries the same sensitivity, residency, and audit obligations as the documents it indexes, because its contents are recoverable. Teams that write these three decisions down before the pilot ships spend their security review defending choices; teams that don't spend it re-architecting.