Debug silent deadlocks in Microsoft Agent Framework workflows by preventing loop jams, controlling agent handoffs, reducing token waste, and improving multi-agent reliability
The Multi-Agent Loop Jam: Debugging Silent Deadlocks
in Microsoft Agent Framework Workflows
Nothing crashes. No exception is thrown. No alert fires. Your triage agent hands off to refund, refund hands back to triage, and the two of them politely pass the same ticket between each other several hundred times while your token bill climbs. This is the loop jam — and because it is silent, the fix is never "catch the error." It is to bound the workflow before it ever starts.
# There is NO stack trace. That is the entire problem. What you see instead
# is a repeating cycle in the OpenTelemetry GenAI trace, and a flat token graph:
TIMESTAMP SPAN agent.name duration
09:14:02.114 invoke_agent triage triage 1,204 ms
09:14:03.318 execute_tool handoff_to_refund triage 12 ms
09:14:03.330 invoke_agent refund refund 1,890 ms
09:14:05.220 execute_tool handoff_to_triage refund 11 ms <-- back
09:14:05.231 invoke_agent triage triage 1,150 ms <-- again
09:14:06.381 execute_tool handoff_to_refund triage 12 ms
09:14:06.393 invoke_agent refund refund 1,932 ms
... (repeats 300+ times, identical, forever)
# If — and only if — a bound was configured, you finally get an honest failure:
WorkflowRunError: workflow did not converge: reached max_iterations (limit=25)
# With no bound configured, the run never terminates. It just spends.Symptom: A workflow run that never returns, with steadily climbing token spend and no exception. Failure point: A cycle in the handoff graph — agent A can hand off to B, and B can hand back to A, with no condition that terminates the exchange. Default platform behaviour: The Agent Framework Runner executes a Pregel-style superstep loop that continues until convergence — meaning no more messages are in flight — or until max_iterations is reached. A ping-ponging pair of agents never stops producing messages, so it never converges. If you did not set a bound, there is nothing to stop it.
Distributed systems engineers have a well-developed instinct for deadlock: two threads, two locks, neither yields, everything stops. A multi-agent loop jam is stranger and, in one important way, worse. Nothing stops. The system is alive — agents are reasoning, tools are firing, spans are streaming into your tracing backend, the workflow reports itself as healthy and running. It is simply not making any progress, and every lap of the loop costs you real money in tokens. There is no exception to catch, no timeout to fire, no failed health check. The workflow will happily do this until you kill the process or your quota runs out. Debugging it starts with accepting that you are not looking for an error — you are looking for a cycle.
The term is borrowed, and slightly wrong — which is useful, because the difference is the whole diagnosis. A classical deadlock halts. A multi-agent loop jam runs forever. It is closer to a livelock: the system is fully alive, doing work, making calls, and getting precisely nowhere.
To understand why nothing stops it, you need one fact about how Agent Framework executes a workflow. The Runner implements a Pregel-style superstep loop: in each superstep it delivers all pending messages to their target executors, runs those executors concurrently, and collects whatever new messages they emit. It repeats this until one of two things happens:
- Convergence — a superstep produces no new messages, so there is nothing left to deliver. The workflow is done.
- max_iterations is reached — an explicit bound stops the loop.
Now consider two agents in a handoff cycle. Triage emits a message to refund. Refund emits a message back to triage. Every superstep always produces a new message. Convergence is never reached. And if you did not configure an iteration bound, the second exit condition does not exist either. The loop has no way out — not because of a bug in the framework, but because you built a graph with a cycle and gave the runtime no instruction about when to stop traversing it.
A crash is loud, fast, and free. A loop jam is quiet, slow, and metered. Your monitoring shows a healthy service. Your workflow reports "running." And every lap of the cycle invokes every agent in it, each consuming its own context window and billing its own tokens independently. A four-agent pipeline uses roughly four times the tokens of a single agent — and a jammed four-agent cycle keeps doing that, indefinitely. Teams routinely discover this from a billing alert rather than an application alert.
Not every jam has the same cause, and the four have genuinely different fixes. Identify which one you have before you reach for a bound, because bounding a workflow that is jamming for reason 3 just converts an infinite loop into a truncated failure — the underlying agent is still confused.
| Jam type | What it looks like in the trace | Root cause | Primary fix |
|---|---|---|---|
| Ping-pong | A → B → A → B, identical spans, forever | A cycle in the handoff graph with no termination condition | Termination condition + max_iterations |
| Hot potato | A → B → C → A, a longer ring, no repetition of content | Vague agent instructions — no agent believes the task is theirs | Sharpen instructions; make boundaries explicit |
| Stall | Same agent invoked repeatedly, output barely changes | The agent cannot make progress but keeps being re-selected | Magentic max_stall_count → auto-replan |
| Tool loop | One agent, the same tool call repeating with the same args | Agent retries a failing tool forever, or re-reads state it never changes | Tool-level retry cap + state clearing |
The "hot potato" jam is the one that fools people, because it looks like a framework problem and is not. In handoff patterns, agents decide when to transfer based on their instructions. Vague instructions lead to agents trying to hand everything on rather than handle it — or, just as bad, trying to handle everything themselves. Be explicit about boundaries: "If the customer asks about X, transfer to Y_agent; otherwise resolve it yourself and end the conversation." No iteration bound fixes an agent that does not know what its job is; it just caps how long it flails.
There is a widespread misconception worth clearing up before any code, because it sends people to the wrong library and wastes days: azure-ai-projects does not provide loop control. It has no step counter, no iteration cap, no termination primitive. If you go looking for one there, you will not find it.
The two SDKs do genuinely different jobs, and you need both:
| Package | Its actual role here | What it gives you |
|---|---|---|
| agent_framework | The fix. All loop control lives here | max_iterations on the Runner; termination_condition on HandoffBuilder; max_round_count / max_stall_count / max_reset_count on MagenticBuilder; checkpointing |
| azure-ai-projects | The microscope. How you see the loop | AIProjectClient; project.telemetry.get_application_insights_connection_string() to wire OpenTelemetry traces into App Insights |
| azure-monitor-opentelemetry | The exporter | configure_azure_monitor() — ships the GenAI spans that reveal the cycle |
| asyncio / CancellationToken | The backstop | Wall-clock timeout around the workflow run — the bound that holds when all others fail |
Use azure-ai-projects to find the loop — it connects your traces to Foundry observability so the repeating A→B→A pattern becomes visible in Application Insights. Then use agent_framework to bound the loop — its orchestration builders are where every guardrail actually lives. Diagnosis and remediation are in different packages, and conflating them is why so many teams get stuck.
Architectural Topology: Failing vs Remediated
Every jammed multi-agent workflow is missing the same four things. This is the target state the fixes below construct.
| Control | Failing configuration (current) | Remediated configuration (fix) |
|---|---|---|
| Iteration bound | None — Runner loops until convergence, which never comes | max_iterations set on the workflow Runner |
| Termination condition | Absent — no function decides when the conversation is done | termination_condition passed to HandoffBuilder |
| Stall detection | None — a stuck agent is re-selected indefinitely | Magentic max_stall_count triggers auto-replan |
| Wall-clock timeout | None — the run can outlive the request, the pod, the day | asyncio.wait_for(...) / CancellationToken around the run |
| Agent instructions | Vague — "help the customer" — so nobody owns the task | Explicit boundaries and an explicit "you may end the conversation" |
| Observability | No tracing — the loop is invisible until the bill arrives | OTel GenAI spans → App Insights via AIProjectClient |
| Cost guard | Unbounded token spend per run | Alert on runs exceeding an iteration/token threshold |
You cannot fix a cycle you cannot see, and a loop jam produces no error to look at. So the first move is always instrumentation. Agent Framework emits traces, logs, and metrics according to the OpenTelemetry GenAI semantic conventions, which means every agent invocation and every tool call becomes a span with standard attribute names. That is exactly what you need — the repeating invoke_agent / execute_tool pattern is the bug, rendered as data.
This is where azure-ai-projects earns its place: AIProjectClient hands you the Application Insights connection string attached to your Foundry project, so you do not have to plumb it manually.
Once spans are flowing, this KQL finds the jam. You are not looking for errors — you are looking for repetition.
Agent Framework can record message content in traces, and it is genuinely useful when debugging a loop — you can see the agents talking past each other. But content recording captures prompts, responses, and function-call arguments. Enable it in development or a scrubbed staging environment, not in production, or you will pipe customer PII straight into Application Insights. Diagnose the loop with content on in a repro; run production with metadata-only spans.
This is the backstop, and it should be non-negotiable on every workflow you ship. It does not make your agents smarter — it converts an infinite loop into a bounded failure, which is the difference between an incident and a log line. Set it even when you are confident your graph is acyclic.
Recall the mechanic: the Runner loops supersteps until convergence or until max_iterations is reached. Setting it gives the second exit condition an actual value.
Pick max_iterations from your workflow's real worst case — count the longest legitimate path through your graph and add headroom, rather than reaching for a round number. Then treat every hit as a defect, not a routine outcome: a workflow that regularly terminates at its iteration cap is not "protected," it is broken and being truncated. Alert on the cap being reached and go fix the underlying cycle.
An iteration cap is a blunt instrument — it stops the loop but knows nothing about whether the work is done. A termination condition is the semantic fix: a function that examines the conversation and decides when the workflow should end. This is the mechanism the framework provides specifically to prevent infinite loops and ensure conversations reach a natural conclusion.
Crucially, HandoffBuilder accepts it as a first-class parameter. If you built a handoff workflow and left it out, this is your bug.
A condition that looks for CASE_RESOLVED is useless if no agent has been told to emit it. This is the single most common way this fix silently fails. Your agent instructions must close the loop explicitly — for example: "When the customer's issue is fully resolved, reply with CASE_RESOLVED and do not hand off to another agent." The termination condition and the agent instructions are one design, not two. Write them together.
Handoff workflows run in autonomous mode by default — agents operate without waiting for user input between turns, which is exactly what lets a cycle run unattended. Switching the interaction mode to interactive pauses for human input during the conversation, which naturally bounds the loop with a human in it. It is not right for every workflow, but for anything expensive or irreversible it turns an unattended jam into a prompt.
The stall jam is different from the ping-pong, and no iteration cap addresses it properly. A stall is when the same agent keeps getting selected and keeps producing near-identical, non-advancing output. The loop is not bouncing between agents — it is grinding in place.
Magnetic orchestration is built for exactly this. A manager agent turns the task into a plan, assigns work to specialists, checks progress after each round, and revises the plan when the team stops moving. It ships with three distinct bounds, and they guard three genuinely different failure modes.
| Guardrail | What it bounds | What happens on hit |
|---|---|---|
| max_round_count | Total rounds of manager-directed work | Hard stop — the ceiling on the whole orchestration |
| max_stall_count | Consecutive rounds with no detectable progress | Triggers an automatic re-plan rather than grinding on |
| max_reset_count | How many times the manager may re-plan | Hard stop — prevents infinite re-planning loops |
Note the layering: max_stall_count is a recovery mechanism (it re-plans and tries a different approach), while max_round_count and max_reset_count are termination mechanisms. Without max_reset_count, a manager that stalls, re-plans, stalls again, and re-plans again has simply moved the infinite loop up one level of abstraction.
Magentic supports .with_plan_review(), which emits the manager's plan as a request for human input before execution begins. For a workflow that will spend real money across many agents, reviewing the plan up front is dramatically cheaper than discovering after 300 laps that the manager's plan was circular from the start. It bounds the loop at the design stage rather than at runtime.
Every bound so far is semantic — it depends on the agents, the manager, or the graph behaving as you expect. The wall-clock timeout is the only one that does not. It is a clock. An agent cannot talk its way past it, and it is the bound that holds when your termination condition has a typo in it.
Bound the run in real time
Clear the state, or the next run inherits the jam
A jammed run leaves debris: a conversation thread bloated with hundreds of circular messages, and a checkpoint that encodes the looping state. If you resume from that checkpoint or reuse that thread, you resume the loop. Agent Framework's workflows support checkpointing so state can be saved and resumed — which is a feature for human-in-the-loop pauses and a liability after a jam.
This is the failure that turns one bad run into a bad afternoon. The instinct after a timeout is to resume from the last checkpoint. But the checkpoint is the loop, frozen mid-lap. Resuming it re-enters the cycle with a conversation history that is now even longer — so the retry is slower, more expensive, and equally infinite. If you retry, retry from the original task with a clean thread, and only after you have changed something (a bound, an instruction, a graph edge). Retrying an unchanged jam is just paying twice.
Validation & Verification: Confirm the Fix
The whole failure mode is silence, so "it seemed fine" is not validation. You have to deliberately provoke the loop and prove that each bound catches it. Three steps: force a jam, confirm it terminates, then confirm it terminates loudly.
Be precise about the success criterion, because it is easy to declare victory too early. Bounds do not mean your agents now cooperate correctly — they mean a failure to cooperate is now finite, loud, and cheap instead of infinite, silent, and metered. A workflow that terminates at its iteration cap on every run is not fixed; it is being truncated, and you still have a broken graph or vague instructions underneath. The bounds are a safety net, not the design. Alert on every bound hit and treat each one as a bug to investigate — the day your caps stop firing is the day the workflow is actually correct.
Key Takeaways
Frequently Asked Questions
Related FAVRITE Articles
- Azure OpenAI to Microsoft Foundry: Fixing Private Endpoint & DNS Failures
- How to Fix Azure OpenAI Token Limits: Architectural Patterns for High-Throughput Apps
- Fixing the First-Request Lag: Azure Functions and Container Apps for AI Microservices
- Azure AI Search RAG Runbook