Implement secure document-level access in Azure AI Search for multi-tenant apps using enforced security filters, tenant isolation, and protected vector search retrieval.
Strict Isolation: Implementing Document-Level Security
in Azure AI Search for Multi-Tenant Apps
A shared vector index is a shared vector index, regardless of how confidently your application code claims to filter it. Relevance ranking doesn't know about tenants. Semantic reranking doesn't know about tenants. The only thing standing between User A and User B's confidential documents is whichever filter your code remembered to attach on that particular request — and "remembered to attach, every time, on every code path" is not a security boundary. It's a hope.
# The incident report that starts this conversation, every time:
User: alice@tenant-acme.com (Tenant: ACME Corp)
Query: "What's in our Q3 termination severance policy?"
Response: Grounded in document "hr-policy-2024.pdf"
Actual owner of that document: Tenant "Globex Industries" — a
DIFFERENT customer of the same multi-tenant SaaS product.
# The application code, on inspection, looked completely reasonable:
async def retrieve_context(query: str, user_id: str):
results = search_client.search(
search_text=query,
query_type="semantic",
top=5,
# filter=f"tenant_id eq '{get_tenant(user_id)}'" <-- commented
# out during
# a debugging
# session six
# weeks ago,
# never restored
)
return [r["content"] for r in results]
# No error. No exception. No log line. The query executes perfectly —
# against the ENTIRE shared index, across every tenant, because nothing
# at the SERVICE layer enforces the boundary. The application was the
# only thing standing between tenants, and the application forgot.
# What SHOULD have caught this, at the platform level:
PUT /knowledgesources/tenant-docs-ks?api-version=2026-05-01-preview
{
"searchIndexParameters": {
"baseFilter": "tenant_id eq @@CALLER_TENANT@@" # persisted, cannot
} # be omitted per-callSymptom: A user's RAG chatbot response is grounded in a document belonging to a different tenant — data the user was never granted access to. Failure point: Tenant isolation was implemented entirely in application code as a per-query filter, with no enforcement at the search-service layer. A missed filter on one code path is a full cross-tenant data leak. Default platform behaviour: Azure AI Search returns whatever a query asks for, from whatever documents are in the index, regardless of who's asking — unless a filter constrains it. The service has no innate concept of "tenant" or "user" until you build one into the schema and the query.
Multi-tenant RAG has a specific failure mode that doesn't exist in single-tenant search: the index itself doesn't care who's asking. A vector similarity search finds the nearest neighbors to a query embedding regardless of which customer uploaded them. Semantic reranking scores relevance regardless of ownership. If tenant isolation lives only in a filter parameter that some code path forgot to attach — a new endpoint added in a hurry, a debugging session that commented it out, a background job that reused a client without the filter — the result isn't a bug users notice. It's a silent, undetected leak, because the query succeeds, the response looks plausible, and nothing throws an exception. The fix that actually holds under this pressure separates the two things that were tangled together in application code: a tenant boundary that is structurally difficult to omit, and a user-level trim that composes on top of it without ever being able to loosen it.
The pattern that leaks is almost always the same shape: a developer adds a filter parameter to the search call, tests it, ships it, and it works — for that code path. The problem surfaces later, on a code path nobody was thinking about when the filter was designed: a new admin dashboard that reuses the same search client without remembering the filter; a background enrichment job that queries the index directly; a debugging session where someone commented the filter out "just to see the full result set" and forgot to restore it; a second developer, six months later, adding a new retrieval function and copying an older one that predates the filter.
None of these are exotic mistakes. They are the ordinary failure mode of a security control that lives as a convention rather than a structural property — every single call site has to remember to do the right thing, correctly, forever, and the control fails silently the moment any one of them doesn't.
| Failure mode | Why it happens | Why "just review the code" doesn't fully fix it |
|---|---|---|
| New code path added without the filter | Developer copies an older query pattern, or writes a new one from scratch, unaware the filter convention exists | Code review catches what reviewers know to look for. A missing filter often doesn't look wrong — it looks like a working query |
| Filter commented out during debugging | Developer wants to see unfiltered results to diagnose a relevance issue, forgets to restore it | The change is small, easy to miss in a diff, and the code still runs without error |
| Background job bypasses the request-scoped filter logic | Batch/cron jobs often construct their own search client, outside the normal request pipeline where the filter convention lived | These jobs are less frequently reviewed and rarely covered by the same integration tests as user-facing paths |
| Third-party integration calls the index directly | A partner service or internal tool is given search access and builds its own queries | The filter convention is tribal knowledge that doesn't travel with API access |
Ask a specific question about your current architecture: if a brand-new engineer, on their first day, writes a search query against your index without reading any internal documentation, does the query come back correctly scoped to one tenant — or does it return everyone's data by default? If the honest answer is "it returns everyone's data unless they remember to add the filter," you don't have a security boundary. You have a convention that has not failed yet. The fix in this article moves the boundary to a place a new engineer's first query cannot bypass by omission.
Azure AI Search's search index knowledge sources support exactly the split this problem needs, and the distinction between the two properties maps directly onto the distinction between tenant isolation and user-level trimming.
| Property | Where it lives | When it applies | Use it for |
|---|---|---|---|
| baseFilter | Persisted on the searchIndexParameters of the knowledge source itself | Every single retrieve request that uses this knowledge source, automatically | Tenant isolation. The boundary that must never be optional |
| filterAddOn | Supplied in knowledgeSourceParams at retrieve time, per request | Only on requests where the caller explicitly supplies it | User/group-level trimming. The narrower, per-query scope on top of the tenant boundary |
The composition model is what makes this architecturally sound rather than just organizationally tidier: filterAddOn combines with the stored baseFilter using AND logic. A request can only ever narrow what the base filter already permits — it has no mechanism to widen it. Even a badly-constructed or missing runtime filter cannot cause a query to escape the tenant boundary, because that boundary isn't expressed as a parameter the request controls at all.
Microsoft's own release notes describe a full precedence model governing service defaults, knowledge source defaults, and per-request overrides — baseFilter and filterAddOn are the concrete expression of that hierarchy for search index knowledge sources specifically. Understanding it as a precedence chain, not two unrelated settings, is what makes the design choice obvious: put the thing that must never be skipped at the layer nearest the data, and the thing that legitimately varies per-caller at the layer nearest the request.
Both filters need something to filter on. Before writing either baseFilter or filterAddOn expressions, the index schema needs two filterable fields designed for exactly this purpose.
| Field | Cardinality | Populated from | Used in |
|---|---|---|---|
| tenant_id | One value per document | Your application's tenant/customer identifier at ingestion time | baseFilter — the persistent boundary |
| group_ids | Collection — a document can belong to several groups | Entra ID group object IDs the document should be visible to, set at ingestion | filterAddOn — the runtime, per-caller trim |
Microsoft's own security filter guidance for Azure OpenAI On Your Data uses group_ids as the default field name specifically for this pattern: a Collection(Edm.String), filterable, holding the Entra ID group object IDs permitted to see that document. You can rename it, but keeping the convention makes onboarding new engineers and support staff faster — and every internal Microsoft doc, sample, and troubleshooting guide assumes it by default.
Microsoft's document-level access guidance is explicit on this point: for ACL-secured content, prefer group access over individual user access, for ease of management. A document tagged with 3-4 group IDs that map cleanly to Entra ID security groups (e.g., grp-hr-benefits, grp-managers) is dramatically easier to keep correct over time than a document re-tagged every time an individual employee joins or leaves a team. Group membership changes happen in Entra ID; your index doesn't need to know about it at all if the filter is built from the token's current group claims at query time.
Architectural Topology: Failing vs Remediated
| Layer | Failing configuration (current) | Remediated configuration (fix) |
|---|---|---|
| Tenant boundary | A filter string built and passed per-call in application code | baseFilter persisted on the knowledge source — not a per-call parameter |
| Group-level trim | Mixed into the same ad-hoc filter string, if present at all | filterAddOn built from the caller's live Entra ID group token claims |
| New code paths | Must remember to reconstruct the full filter from scratch | Automatically tenant-scoped via baseFilter; only need filterAddOn for finer trim |
| Background jobs / direct API callers | Bypass the filter convention entirely if they build their own client | Still tenant-scoped — the boundary lives on the knowledge source, not the caller's code |
| Schema | No dedicated security fields, or an inconsistent ad-hoc field | tenant_id (single) + group_ids (collection), both filterable |
| Failure mode of a bug | Missing filter = full cross-tenant leak | Missing filterAddOn = over-broad within one tenant, never across tenants |
| Group membership changes | Requires re-tagging documents or re-indexing | Resolved live from the caller's token at query time — no index changes needed |
| Native ACL option | Not evaluated — custom filter is the only mechanism considered | Considered alongside filters for ADLS Gen2 / SharePoint sources with existing ACLs (Section 7) |
This is the structural fix — the one that turns tenant isolation from a convention into a property of the knowledge source itself. Set it once, at knowledge source creation (or update), and every subsequent retrieve request through that knowledge source is bound by it, regardless of which code path issued the request.
Notice the shape: baseFilter is a fixed string on the knowledge source, not a template with a placeholder that gets filled per-request. That means the clean architecture is one knowledge source per tenant, provisioned when the tenant onboards, each with its own hardcoded tenant_id eq '...' baseFilter pointing at the shared underlying index. Your application looks up which knowledge source belongs to the authenticated caller's tenant and always retrieves through that one. There's no runtime string interpolation of the tenant ID into a filter — the tenant boundary is provisioned infrastructure, not request-time logic.
Even though baseFilter is set at provisioning time rather than per-request, it is still constructed by string interpolation somewhere in your onboarding code. Validate that tenant_id is drawn from your own system (a GUID or slug you generated, not arbitrary user input) before it goes into the filter string, to avoid OData filter injection at the one place this pattern still builds a string dynamically.
With the tenant boundary structurally guaranteed by baseFilter, filterAddOn handles the narrower, legitimately-per-request concern: which groups within that tenant the specific caller belongs to. This is where the caller's Entra ID group membership token claims get translated directly into an OData filter expression.
The group list passed into build_group_filter must come from a source the caller cannot influence — the validated Entra ID access token's groups claim, or a server-side Microsoft Graph memberOf call keyed off the token's verified subject. It must never be a field the client sends in the request body ("here are my groups: ..."). If your application accepts client-supplied group lists, the entire filter — however cleanly built — is decorative, because any caller can simply claim membership in whatever group unlocks the data they want.
Entra ID tokens have a practical limit on how many groups can be embedded directly in the groups claim. When a user belongs to more groups than fit, the token instead includes an overage indicator, and your application must call Microsoft Graph's memberOf endpoint to resolve the full list server-side. If your build_group_filter function only ever reads the token's inline groups claim, users in many groups will silently see an incomplete filter — narrower than intended, which fails safe in this direction, but will also generate confusing "why can't I see this document I should have access to" support tickets. Check for the overage indicator and fall back to Graph explicitly.
The filter-based pattern above is the general-purpose fix, and it works for any data source. But if your documents originate from a system that already has real ACLs — ADLS Gen2 containers and files, or SharePoint in Microsoft 365 — Azure AI Search has a more native mechanism worth knowing about: it can validate the caller's Microsoft Entra token directly against synchronized document ACL metadata, trimming results without you writing any filter expression at all.
| Aspect | Custom filter (baseFilter/filterAddOn) | Native token-based ACL |
|---|---|---|
| Data source fit | Any source — you own the schema and tagging entirely | ADLS Gen2 (native), SharePoint (native) — sources with real, syncable ACLs |
| Setup effort | Design fields, build filter logic, maintain group tagging at ingestion | Lower — permissions sync from the source system automatically |
| Permission drift | You must keep group_ids current with reality yourself | Synced from the source of truth (ADLS Gen2 ACLs, SharePoint permissions) |
| Custom access models | Full flexibility — model tenant + group + any custom dimension you want | Limited to what the source system's native ACL model expresses |
| Multi-tenant SaaS with app-level tenancy | The right fit — "tenant" usually isn't a concept ADLS Gen2 or SharePoint ACLs express directly | Awkward fit unless tenancy maps cleanly onto folder/site structure |
The native ACL path is compelling when your documents genuinely originate from ADLS Gen2 or SharePoint with real, actively-maintained ACLs — you inherit correctness from a system that's already the source of truth for permissions. But "tenant" in a multi-tenant SaaS product is usually an application-level concept that doesn't map cleanly onto blob container ACLs or SharePoint site permissions. For that shape of problem — which is what this article is centrally about — the baseFilter/filterAddOn pattern remains the more direct fit, because you're modeling a concept (tenant) the underlying storage system doesn't natively have.
The baseFilter/filterAddOn pattern assumes a shared index across tenants, which is the right default for most multi-tenant SaaS — one index, filtered per tenant, is dramatically simpler to operate than provisioning and maintaining an entire index per customer. But it's worth being explicit about when the shared-index assumption stops being the right one, because for a subset of customers it isn't.
| Approach | Best for | Trade-off |
|---|---|---|
| Shared index, per-tenant knowledge source + baseFilter | Most multi-tenant SaaS — many small-to-medium tenants, self-serve onboarding | Operationally simple; isolation is logical (filter-enforced), not physical |
| Index-per-tenant | A small number of large, high-value, or regulatorily sensitive tenants (e.g. a healthcare or financial services customer with contractual data-isolation requirements) | Physical isolation — no shared storage at all. Higher operational overhead: N indexes to manage, monitor, and scale |
| Hybrid — shared index for most, dedicated index for a few | SaaS products with a tiered customer base and a genuine enterprise/regulated tier | Most complexity, but matches risk to isolation cost — don't pay physical-isolation overhead for every tenant if only a few need it |
The honest engineering answer is that a correctly-implemented baseFilter boundary provides strong logical isolation — a bug cannot make one tenant's data appear in another tenant's results, for the structural reasons Figure 2 lays out. Index-per-tenant is usually not chosen because the filter pattern is insecure; it's chosen because a specific customer's contract, compliance framework, or internal risk policy requires physical rather than logical separation, and no amount of correct filter logic satisfies that requirement on paper. Know which of your tenants actually need that guarantee before defaulting everyone into the higher-overhead architecture.
Because tenant isolation bugs are invisible until a specific unlucky query surfaces them, teams often ship patterns that look secure in code review and hold up fine until the one code path nobody checked.
| Anti-pattern | Why it feels right | Why it isn't |
|---|---|---|
| Build the tenant filter as a per-request string in application code | "It's just one more parameter on the query" | Every new code path has to remember to build it correctly. This is the exact pattern that leaked in the opening incident |
| Accept client-supplied group IDs in the request body | "The frontend already knows the user's groups" | Any caller can claim membership in any group. The filter becomes decorative — it enforces whatever the client says, not reality |
| Empty group filter defaults to "show everything" | "If we can't determine groups, don't block the user" | Fails open. A token parsing bug or a Graph API outage becomes a full permission bypass instead of a visible error |
| One knowledge source, tenant ID passed as filterAddOn instead of baseFilter | "Same effect, one less thing to provision" | Moves the tenant boundary back into per-request logic — exactly the structural weakness baseFilter exists to remove |
| Trust the group claim in an unvalidated or expired token | "The token has a groups field, that's enough" | An unvalidated token's claims are just strings an attacker can forge. Verify signature, issuer, audience, and expiry before trusting any claim inside it |
| Test only the happy path (correct tenant, correct groups) | "The main flow works, ship it" | The dangerous bugs live in the paths nobody tests: missing groups, malformed tokens, background jobs, admin overrides. Test those deliberately |
Validation & Verification: Confirm the Fix
The only test that actually proves tenant isolation holds is an adversarial one: deliberately try to see another tenant's data, from a legitimate user account, and confirm it's impossible — not just unlikely.
Three conditions must hold together. One: every knowledge source used in production has a non-empty baseFilter scoping it to exactly one tenant, verified by reading the knowledge source configuration directly, not by trusting application code comments. Two: the adversarial cross-tenant test — a real, authenticated user from one tenant querying for content known to exist only in another tenant — returns zero results, every time, including after code changes. Three: the group-membership resolution path fails closed on missing or unresolvable data, verified by a dedicated test, not assumed. Miss any of the three and the isolation is real only by accident, which is the same condition that caused the original incident.
Key Takeaways
Frequently Asked Questions
Related FAVRITE Articles
- Splitting the Bill: Isolating Semantic Ranker Costs from Agentic Retrieval Plans
- RBAC for Azure Files: Least-Privilege Access Patterns
- Purging the Keys: Migrating Azure OpenAI Applications to Managed Identities and RBAC
- The Missing Data RAG Pipeline Bug: 30-Second Timeouts in Azure AI Search