Fix Azure AI Search custom skill timeouts that silently drop documents from RAG indexes, causing missing data, indexing failures, and inaccurate retrieval results.
The "Missing Data" RAG Pipeline Bug: Fixing the 30-Second Timeout in Azure AI Search Custom Skills
Your RAG app confidently tells a user it has no information on a document you personally uploaded last week. The indexer says Success. The blob is right there in storage. What actually happened is that a custom skill blew a timeout, the indexer downgraded the failure to a warning, and the document was silently dropped from the index — and it will keep happening on every run until you find it.
Could not execute skill because it did not execute within the time limit '00:00:30'.
This is likely transient. Please try again later. For custom skills, consider
increasing the 'timeout' parameter on your skill in the skillset.
# It appears as a WARNING — not an error — in the indexer execution result:
GET /indexers/my-indexer/status?api-version=2024-07-01
{
"lastResult": {
"status": "success", <-- the indexer says it SUCCEEDED
"itemsProcessed": 487,
"itemsFailed": 13, <-- 13 documents silently never made it
"warnings": [
{
"key": "https://sa.blob.core.windows.net/docs/contract-2024-Q3.pdf",
"name": "myCustomSkill",
"message": "Could not execute skill because it did not execute within
the time limit '00:00:30'."
}
]
}
}
# Downstream, this is all your users ever see:
> "I don't have any information about the Q3 contract."Symptom: Documents you know exist are unretrievable in RAG, while the indexer reports success. Failure point: Indexer → skillset → Custom Web API skill exceeds its timeout → the enrichment fails → the document is skipped. Default platform behaviour: The indexing pipeline is synchronous. If your skill does not respond inside the timeout window, the call is abandoned. And because maxFailedItems is commonly set to a non-zero value, the failure is downgraded to a warning and indexing continues — producing a green status over an index with holes in it.
The cruelty of this bug is that every dashboard lies to you. The blob is in storage. The indexer status says success. The skillset is valid. The search index exists and returns results for other queries. Nothing anywhere is red. And yet a specific set of documents — invariably the largest and most important ones, the hundred-page contracts and the scanned reports — are simply not in the index, and your RAG application answers questions about them by confidently claiming ignorance. The failure is real, it is logged, and it is sitting in a warnings array that nobody reads because the status field above it says the run succeeded.
Before you architect a workaround, know this: the 30-second cap is not fixed, and it is not rigid. A very large number of teams have built elaborate queue-and-callback machinery to escape a limit that they could have raised with a single JSON property.
Microsoft's documentation is unambiguous. By default, the connection to a custom skill endpoint times out if no response is returned within a 30-second window (PT30S). But you can increase that interval to a maximum of 230 seconds (PT230S) by setting the timeout parameter on the skill definition. Most skillsets simply never set it, inherit the default, and their authors conclude the platform is inflexible.
So the honest framing of the problem is this: there is a hard limit, but it is 230 seconds, not 30. That distinction matters enormously, because it changes what kind of problem you are solving.
| Your skill's real execution time | What you actually have | The correct fix |
|---|---|---|
| Under 30s | No problem — you are hitting a different bug | Look elsewhere (auth, response shape, cardinality) |
| 30s – 230s | A configuration problem | Set timeout. Takes five minutes. Do this first |
| Over 230s | An architectural problem | Reduce the work per invocation — chunk before the skill (Section 6) |
Do not go looking for a way past it. 230 seconds is the maximum permitted value for the timeout parameter, and the indexing pipeline is synchronous — the indexer blocks on your HTTP response. There is no async callback, no polling contract, no "come back later" status. If your skill cannot answer within 230 seconds, the answer is not a bigger timeout. It is less work per call. That is what the rest of this guide is about.
A timeout, on its own, is a manageable failure. What turns it into the missing data bug — a bug that survives in production for months — is the indexer's error tolerance setting.
maxFailedItems tells the indexer how many documents may fail before the whole run is declared a failure. It is very commonly set to a non-zero value (often -1, meaning "unlimited"), because on a large corpus you do not want one corrupt PDF to abort a six-hour indexing run. That is a reasonable instinct. It also means that every skill timeout is downgraded from an error into a warning, the affected document is skipped, and the run reports "status": "success".
| maxFailedItems | What happens on a skill timeout | Do you find out? |
|---|---|---|
| 0 | The indexer run fails immediately | Yes — loudly, in status and alerts |
| > 0 (e.g. 10) | Document skipped; counted toward the budget; run continues | Only if you read the warnings array |
| -1 (unlimited) | Every failing document is silently skipped, forever | No. This is the missing-data bug |
The consequence is a search index that is quietly incomplete. And the documents it drops are not random — they are systematically the largest and most content-rich ones, precisely because size is what causes the timeout. Your RAG system ends up missing exactly the documents your users most want to ask about: the long contracts, the detailed reports, the scanned archives.
This is the part worth sitting with. A random data-loss bug would be annoying. This one is biased: it discards documents in direct proportion to how much content they contain. The 400-page master services agreement times out; the two-page memo sails through. So your index is fullest exactly where the content is thinnest, and emptiest exactly where it matters most — and the failure presents as your AI being uselessly ignorant about your most important material.
Before tuning anything, you must know how many times your skill is actually invoked per document — because it determines everything about how you fix this, and it is set by one easily-overlooked property: context.
| context value | Invocation cardinality | Payload per call |
|---|---|---|
| /document | Once per document | The entire document. A 300-page PDF arrives in a single call — this is what blows the timeout |
| /document/content | Once per document | Whole content field — same problem |
| /document/pages/* | Once per page/chunk | One chunk. Small, fast, predictable — this is the fix |
Microsoft's own scaling guidance makes this the first thing to establish: know whether the skill executes once per document or multiple times per document, because if it runs many times per document you should stay on the lower side of batchSize and degreeOfParallelism to reduce churn.
If your custom skill's context is /document or /document/content, you are handing your entire 300-page document to one HTTP request and asking it to finish in under 30 seconds. No timeout value fixes that at scale — it just moves the wall from 30s to 230s and buys you slightly bigger documents before the same failure returns. The real fix is to change the cardinality, so the skill is invoked many times with small payloads instead of once with an enormous one. That is Section 6.
Architectural Topology: Failing vs Remediated
| Component | Failing configuration (current) | Remediated configuration (fix) |
|---|---|---|
| Skill timeout | Unset — silently inherits the PT30S default | Explicitly set, sized to the real p99 of the skill |
| Skill context | /document — the whole document in one call | /document/pages/* — one small chunk per call |
| Chunking | Inside the skill, or not at all | Before the skill, via SplitSkill |
| Payload per invocation | Unbounded — grows with document size | Bounded and predictable, regardless of document size |
| batchSize | Default — many docs per request, compounding the timeout | Tuned; lowered when cardinality is high |
| maxFailedItems | -1 — every failure silently swallowed | 0 in CI; a small, alerted budget in production |
| Failure visibility | Warnings array nobody reads; status says success | Alert on itemsFailed > 0 and on warning count |
| Long-running work | Executed inline, inside the synchronous skill call | Dispatched — the skill returns fast; work happens elsewhere |
Do this first, because it costs five minutes and it may be all you need. But be clear-eyed about what it is: headroom, not a solution. Raising the timeout does not make your skill faster; it just lets it be slow for longer before the same wall arrives.
Note the pairing of timeout and batchSize. Microsoft's guidance is explicit: if your custom skill cannot execute consistently within 230 seconds, reduce the batchSize so it has fewer documents to process in a single execution. These two knobs work together — a large timeout with a large batch size just means you fail slower.
Do not simply set PT230S everywhere and move on. An over-long timeout is its own problem: it holds indexer threads open, slows the whole pipeline, and masks a skill that is quietly degrading. Measure your skill's real p99 execution time under production-shaped payloads, then set the timeout to that plus a sensible margin. If the honest answer to "what is my p99?" is "more than 230 seconds," you have an architecture problem and no amount of configuration will save you — go to Section 6.
This is the fix that actually solves the problem, and it is a change in where the chunking happens rather than whether it happens. The instinct is to make the skill handle oversized payloads — to add chunking logic, streaming, or batching inside the custom skill. That is the wrong layer. The skill is downstream of the problem; by the time the payload arrives, the timeout clock is already running.
Instead, put a SplitSkill in front of your custom skill, and then set your custom skill's context to iterate over the resulting chunks. This inverts the economics completely: instead of one call that must process 300 pages in 30 seconds, you get 300 calls that each process one page in under a second.
The document size is now decoupled from the skill's execution time. A two-page memo produces two calls; a 300-page contract produces 300 calls. Each individual call is small, fast, and nowhere near any timeout. Crucially, the skill's execution time no longer grows with document size — which is what made the old design fail on exactly your most important documents.
There is a bonus here that has nothing to do with timeouts. RAG needs chunked content anyway — you cannot embed a 300-page document as a single vector and expect useful retrieval. So chunking before the skill is not a workaround you are grudgingly adopting to dodge a limit; it is the shape a RAG pipeline should have had in the first place. The timeout was the symptom that forced you to discover the correct architecture. Set pageOverlapLength so chunks overlap slightly, or you will sever sentences at chunk boundaries and degrade retrieval at exactly those seams.
Changing context to /document/pages/* changes the shape of your enrichment tree — your skill's output is now an array of per-chunk results, not a single value. Your output field mappings must be updated accordingly, and if you are projecting to a knowledge store or an index with one-document-per-chunk, your index schema likely needs to change too. This is the part of the migration that bites people who copy the skillset change without following through. Test on a single document before you reset the indexer.
Once cardinality is right, throughput becomes a tuning exercise across three interacting knobs. There is no one-size-fits-all recommendation — Microsoft says so explicitly, and you should plan to test configurations rather than copy someone else's numbers.
| Knob | Where it lives | What it controls |
|---|---|---|
| batchSize (skill) | Skill definition | How many records are packed into one HTTP request to your skill |
| degreeOfParallelism | Skill definition | How many concurrent requests the indexer makes to your skill |
| batchSize (indexer) | Indexer definition | How many documents are read from the data source and enriched concurrently |
The strategy divides into two shapes: fewer large requests, or many small requests. Which one you want depends on your skill. The critical rule, straight from Microsoft's scaling guidance: if your skill executes multiple times per document — which, after Fix 2, yours now does — stay on the lower side of batchSize and degreeOfParallelism to reduce churn, and consider raising the indexer batch size for more scale instead.
The skill's batchSize and the indexer's batchSize must be coordinated, not tuned in isolation. A common failure is an indexer pulling documents far faster than the skill can drain them, which builds a queue, inflates per-call latency under load, and reintroduces the very timeouts you just eliminated — this time only under production traffic, which makes it look intermittent and "transient." If the errors only appear at scale, this is where to look.
Every fix above reduces the chance of a timeout. This one ensures that if a timeout ever happens again, you find out — instead of discovering it months later through a user complaint. It is the difference between a bug and a silent, permanent data-integrity defect.
The most reliable detection is also the simplest: compare the number of source documents to the number of indexed documents. If blob storage has 500 documents and your index has 487, you have lost 13 — regardless of what any status field claims. Run this reconciliation on a schedule and alert on any drift. It catches not just skill timeouts but every other silent-drop failure mode in the pipeline, including ones nobody has thought of yet.
Occasionally the work per irreducible unit genuinely exceeds 230 seconds — a heavyweight ML model, an OCR pass over a huge scanned image, an external API that is simply slow. You cannot chunk your way below the wall, and the synchronous pipeline will not wait. In that case, stop trying to do the work inside the skill at all.
The pattern is to make the custom skill a fast dispatcher rather than a worker. It does not perform the expensive computation — it looks up an already-computed result, and the expensive work happens outside the indexing pipeline entirely.
The heavy processing runs as a separate pipeline — a Durable Function, a Container Apps job, a Service Bus consumer — triggered when the document lands in storage, long before the indexer ever sees it. By the time the indexer runs, the answer is already sitting in the store waiting to be read.
Notice what has happened architecturally: the expensive work has moved upstream of the indexer, into the ingestion path. That is the correct home for it. The indexing pipeline is a synchronous enrichment system with a hard 230-second budget per call — it is not, and was never intended to be, a general-purpose batch compute platform. If you are fighting its timeouts, you are usually asking it to be one. Move the heavy lifting to ingestion, pre-chunk and pre-compute there, and let the skillset do what it is good at: fast, bounded enrichment.
Validation & Verification: Confirm the Fix
Because the failure is silent, "the indexer says success" proves nothing — it said that while the bug was live. Validation must prove a document count reconciliation, not a status code. Three steps.
Three conditions, all required. One: the source document count equals the indexed document count — this is the only claim that cannot be faked by a green status field. Two: the skill's execution time is now independent of document size, so the failure cannot silently return as your corpus grows. Three: a future timeout would fail the run rather than emit an unread warning. If you fix the timeout but leave maxFailedItems: -1 in place, you have not fixed the missing-data bug — you have only made it rarer, and you will lose the next document just as quietly as you lost the last one.
Key Takeaways
Frequently Asked Questions
Related FAVRITE Articles
- Azure AI Search RAG Runbook
- How to Fix Azure OpenAI Token Limits: Architectural Patterns for High-Throughput Apps
- The Edge Latency Drop: Offloading Embeddings to Foundry Local 1.2
- Fixing the First-Request Lag: Azure Functions and Container Apps for AI Microservices