RAG without data leakage: per-product and per-tenant retrieval

Retrieval-augmented generation — RAG — is the standard way to make a language model answer from your own material. You store documents in a searchable index, retrieve the most relevant passages for each question, and give them to the model as context. The model answers from what it was handed.

That last sentence is also the problem. A model will use anything in its context. If retrieval hands it a document the user should not see, no system prompt can reliably stop that document from shaping, or appearing in, the answer.

This note explains how we build RAG without data leakage across Oryvelon's companies: separate indexes per product, tenant separation where products serve organisations, access filtering before retrieval, and keeping indexes in step with the source of truth. It sits alongside our broader principle of shared infrastructure, separate data.

Where RAG leaks

It helps to be precise about how leakage happens, because most of it is not the model misbehaving.

Retrieval returns the wrong documents. The index contains material from several users, tenants or products, and the query is not filtered. The most relevant passage belongs to someone else. The model uses it. This is the most common case by far.

Metadata filters are optional. The filter exists, but a code path forgets to pass it — a new feature, an admin tool, a background job. The index happily returns everything.

Filters come from the wrong place. The tenant or user identifier used for filtering is taken from something the user controls: the prompt, a request parameter, a field the model extracted. Anyone who can change it can read other people's material.

The index outlives the source. A document is deleted or its permissions change in the product, but the copy in the index remains. Retrieval keeps serving it.

Prompts are treated as access control. The system prompt says "only use documents belonging to this customer". The model mostly complies. Mostly is not a security property.

Logs and caches hold the context. Even if retrieval was correct, full prompts with retrieved passages are logged or cached somewhere broader than the original documents.

Every one of these is a design problem, and every one can be closed by design.

Our rule: no group-wide knowledge

The simplest decision we made was also the most important. There is no Oryvelon knowledge base. No shared index, no group-level document store, no "company brain" that products query.

Each company has its own:

• document sources and ingestion jobs;
• index or indexes, in its own storage;
• embedding and model credentials, under its own AI project;
• retrieval configuration and prompts, registered under that product;
• logs and retention rules.

The retrieval code is shared. It is a library that every product can use, and the AI gateway applies the usual routing, budgets and validation. But the library is always configured with one product's index and one product's credentials. It has no way to query across products because nothing it can reach spans them.

Why so strict? Because the companies are genuinely different businesses with different users and different obligations. MerchNivo's knowledge about a merchant's store has no place in a ZodiVela reading. EduRelia's school materials have no place in CastLyra's brief drafting. And each company is designed so it could be separated from the group and keep running; see Designing every company to stand alone. An index shared across companies would make that separation a forensic exercise.

Three kinds of knowledge, three levels of separation

Within a single product, not all retrieved material is equally sensitive. We sort it into three kinds.

Most leaks come from mixing these kinds in one index with weak filtering. Keeping them apart structurally — different indexes or namespaces, not just a field on each record — removes whole categories of mistake.

Separate indexes or a filtered shared index?

For tenant knowledge, there are two common designs.

Separate index or namespace per tenant. Each school, store or brand gets its own. Retrieval is pointed at the tenant's index by the application, based on the authenticated session. A query cannot return another tenant's documents because it never touches them. Deleting a tenant is deleting its index.

One index with a mandatory tenant filter. All tenants' documents live together, each tagged with a tenant identifier. Every query must include a filter on that identifier.

The second design can be efficient and is widely used. Its weakness is the word "must". If any code path forgets the filter, isolation fails silently. Where we use it, the filter is not a parameter the calling code adds; it is applied inside the retrieval library from the session context, and the library refuses to run a query without it.

Our default is the first design where the number of tenants is manageable and deletion matters, and the second only with that enforced filter. See Tenant isolation explained for the broader pattern.

Filters come from the session, never from the prompt

This is the rule we repeat most often in code review.

The identifiers used to scope retrieval — product, tenant, user, role — are taken from the authenticated session on the server. They are never taken from:

• the user's question or any text they typed;
• a request parameter the client can change;
• anything the model produced, such as an extracted "customer name".

Consider a MerchNivo merchant who types: "Compare my return policy with the one used by the store at example-store.com." If the store identifier for retrieval came from the text, the merchant could read another store's policy documents. Because it comes from the session, the retrieval only ever looks inside the current store's own material. The model answers that it has no information about other stores — which is exactly right.

The same logic applies to roles within a tenant. On EduRelia, a teacher and a student in the same school do not see the same material. The role comes from the session, and it is part of the filter.

Access filtering before the model, not after

Some systems retrieve broadly and then ask the model to ignore what the user should not see, or filter the answer afterwards. Both are weak.

Once a document is in the context, it can influence the answer even if not quoted. A summary can reflect it. A tone can reflect it. A refusal can reveal that it exists. And post-filtering an answer for "leaked content" requires knowing what leaked, which is hard.

So the order is fixed:

1. Resolve identity and permissions from the session.
2. Build the retrieval query with mandatory filters for product, tenant and role.
3. Retrieve.
4. Re-check each result's permissions against the session (a cheap second check that catches stale metadata).
5. Only then place results in the prompt.

Step 4 sounds redundant. It has caught real inconsistencies in our testing, such as a document whose permissions were changed in the product but whose index metadata had not yet updated.

Keeping the index in step with the truth

An index is a copy. Copies drift. For RAG to be safe, the index has to follow the source of truth closely.

Deletions. When a user deletes a document, a school removes a lesson, or a store closes its account, the corresponding index entries must be removed as part of the same deletion flow, not in a monthly clean-up. We treat the index as one of the places personal and tenant data lives. See Data retention and deletion.

Permission changes. If a document moves from "all staff" to "leadership only", the metadata in the index must change too. Until it does, the re-check in step 4 is the safety net.

Content updates. For KeşifAtlası, the rules database is the source of truth for visa and relocation eligibility. When a rule changes, the old text must stop being retrieved immediately. Each indexed passage carries the rule identifier and version; retrieval only returns passages whose version matches the current one in the rules database. An out-of-date passage is simply not eligible. See Rules engines for eligibility.

We also reindex on a schedule and compare counts between source and index. A mismatch triggers a check. It is a dull control and a useful one.

What goes into the index in the first place

The safest data in an index is data that was never put there.

Before indexing, we ask what the feature actually needs. For a help assistant, it needs help articles, not customer tickets. For a store's policy assistant, it needs policy pages and product information, not order histories with customer names. For EduRelia, it needs lesson content, not student work — and where student work is used, it stays in the student's scope and follows the product's child data rules. See Data minimisation for children in edtech.

Where source documents contain personal details the feature does not need — names, emails, phone numbers in a support document, for example — they are removed or masked at ingestion. CastLyra is a good example: talent contact details are governed by explicit access rules on the marketplace, so they are never indexed for AI features at all. The model cannot reveal what it was never given.

Prompts, logs and caches

Retrieval can be perfect and data can still spread through the surrounding plumbing.

Prompts. Retrieved passages are placed into the prompt clearly marked as reference material, with the source identifiers the validator can check. The prompt tells the model to answer only from those passages and to say when they do not contain the answer. This improves quality; it is not our access control.

Logs. By default the gateway logs metadata — which documents were retrieved (by identifier), token counts, latency, prompt version — not the passages themselves. Where a product needs full content for evaluation, it is sampled, filtered and kept for a short, defined period in that product's own storage.

Caches. A response grounded in tenant or personal documents is either not cached or cached with the tenant and user in the key. Product-level knowledge, such as a help article explanation, can be cached normally. See Latency and caching for AI.

Evaluation sets. Test questions and expected answers for a product's RAG feature belong to that product and use synthetic or consented data. They are never assembled from one product's real users to test another.

Testing for leakage

We test isolation directly, not just answer quality.

• Cross-tenant probes. In staging, create two tenants with distinctive, made-up documents — a fictional return policy with an unusual phrase, for instance. From tenant A's session, ask questions designed to surface tenant B's phrase. Any hit is a failure.
• Missing-filter tests. Call the retrieval library without a tenant context and confirm it refuses rather than returning everything.
• Prompt injection attempts. Ask the assistant, in many phrasings, to ignore its instructions, show all documents or act as another customer. Because scoping does not depend on the prompt, these should fail harmlessly.
• Deletion tests. Delete a document and confirm it stops appearing in retrieval within the defined time.
• Version tests. For KeşifAtlası, update a rule and confirm the old passage is no longer returned.

These tests run whenever the retrieval library or a product's retrieval configuration changes.

A worked example: a school's own materials on EduRelia

EduRelia licenses its learning product to schools, which then offer it to students and families. That B2B2C shape — described in B2B2C licensing for schools — makes it a useful test of every rule above.

Imagine a school that uploads its own revision notes for a history unit, so that the study assistant can answer questions in the school's own terms. Here is how the retrieval is set up.

The school's notes go into that school's own namespace, separate from EduRelia's general lesson content and separate from every other school. Before indexing, the notes are checked for personal details — a teacher's email address at the bottom of a page, a class list pasted by mistake — and anything like that is removed. Each passage is tagged with the school, the unit and the audience: staff only, or staff and students.

When a student in that school asks a question, the server reads the school and the role from the session. Retrieval searches two places: EduRelia's general content for the subject, and the school's namespace, filtered to passages marked for students. It never searches another school's namespace, because the library has no way to address it from this session. Staff-only notes, such as a marking guide, are excluded by the role filter and then excluded again by the re-check.

If the school ends its licence, its namespace is deleted as part of the offboarding flow, along with any cached answers keyed to it. The general lesson content is untouched. Nothing the school contributed lingers in answers for anyone else.

When not to use RAG at all

RAG is not always the right tool, and sometimes skipping it is the safest design.

If the answer depends on exact, current facts — a store's stock level, an eligibility outcome, a booking status — retrieval over documents is the wrong source. Those facts should come directly from the verified system through a query in code, then be handed to the model to explain. That is the core idea in AI explains, verified data decides. An index of stock levels would be stale within minutes.

If the material is small and stable, such as a product's refund policy or a short set of reading guidelines for ZodiVela, it may simply be included in the registered prompt. No index, no retrieval, nothing to filter.

And if a feature would need to retrieve personal data to work, we first ask whether the feature is worth that. Sometimes the answer is to redesign it so the personal part is handled by templates and code, with the model only explaining general material.

Common mistakes

A few patterns worth avoiding:

• One index "for now". Starting with a single index across products or tenants and planning to split it later. Later rarely comes, and splitting a live index is harder than building two.
• Admin tools without filters. Internal tools that query the index directly for debugging, bypassing the library. They should use the same enforced path, with least-privilege access.
• Trusting the model to refuse. "The model is told not to reveal other customers' data" is a hope, not a control.
• Forgetting derived data. Summaries, embeddings and cached answers are all derived from source documents and must be deleted when the source is.
• Mixing public and private in one index. Even with filters, a mistake then exposes the private material. Separate them.

Summary

RAG without data leakage is mostly about the retrieval step, because a model will use whatever it is given. At Oryvelon there is no shared knowledge base: each company has its own sources, indexes, credentials and logs, even when the retrieval code is shared. Within a product, we separate product, tenant and personal knowledge; scope every query with identifiers from the authenticated session rather than the prompt; filter before retrieval and re-check after; keep indexes in step with deletions, permission changes and rule updates; index only what features need; and test isolation directly with cross-tenant probes, missing-filter checks and deletion tests.

Questions and answers

How does data leak in retrieval-augmented generation?

Most RAG leaks happen when the retrieval step returns documents the current user should not see, and the model then quotes or summarises them. Prompt instructions cannot reliably prevent this, so filtering must happen before retrieval results reach the model.

Should multi-tenant AI products use one vector index or several?

Either can work, but tenant filtering must be enforced by the system rather than by the prompt. Separate indexes or namespaces per tenant give stronger isolation and simpler deletion; a shared index requires a mandatory tenant filter on every query.

Can companies in the same group share an AI knowledge base?

At Oryvelon they do not. Each company has its own knowledge base, index and credentials, even when the retrieval code is shared, so that no product's data can appear in another product's answers.

Published by Oryvelon, a digital company builder · About the group · Our companies · More in AI Operations

NextSafety boundaries for consumer AI products: pre and post hooks →