Skip to main content

Fix Azure DevOps pipeline timeout issues with proven troubleshooting approaches and performance improvements

Troubleshooting Azure DevOps Pipeline Timeouts:
5 Real-World Fixes That Work

Your pipeline timed out. The deployment window is closing. Stakeholders are asking questions. This guide covers the five most common real-world causes — with exact error messages, diagnostic commands, YAML fixes, and step-by-step runbooks for each one. No theory. Just the fix.

60 min
Hard ceiling on Microsoft-hosted agents for private repos — non-negotiable, regardless of what your YAML says
3,600
The default timeoutInMinutes for Azure Pipelines jobs — 3,600 minutes (60 hours). If you have not set this explicitly, your pipeline will wait a very long time before failing on timeout
Silent
How agent heartbeat loss fails — no error, no log entry, just silence until the job timeout fires and you get "We stopped hearing from the agent"
5
Distinct timeout failure modes covered in this guide — each with a different root cause, different error message, and a different fix

First: Read the Error Message Correctly — Not All Timeouts Are the Same

Azure DevOps uses the word "timeout" loosely, and treating all timeout failures as the same problem is the fastest way to waste an hour applying the wrong fix. Before you touch the YAML or restart the agent, identify which of the five failure modes you are actually looking at. The error message — and where in the pipeline the failure appears — tells you exactly which section of this guide to jump to.

Figure 1 — Azure DevOps pipeline timeout diagnostic tree: identify your failure mode in 60 seconds
Pipeline run failed with timeoutWhere did it fail?Check: did the job ever START running?(Look at the pipeline run status timeline)Job never started(Queued forever)Job started thentimed out / hungFIX 3 — Parallelism QuotaNo hosted parallelism / agents busy→ Jump to Fix 3Started OKWhich agent type?vmImage: ubuntu/windows vs self-hosted poolvmImage / Microsoft-hostedFIX 1 — 60-Minute Hard LimitJob hit 60:00 on Microsoft-hosted agent→ Jump to Fix 1Self-hosted agentWhat does the log show?Heartbeat lost / ECONNREFUSED / silent hangFIX 2 — Heartbeat Lost"Stopped hearing from agent"→ Fix 2FIX 4 — ProxyECONNREFUSED / 407→ Fix 4FIX 5 — Service Connection ExpiredTask hangs on deploy step — 401 in verbose log
Use this tree before reading any fix section. Identify whether the job queued, started and crashed, or started and silently hung — then jump directly to the relevant fix.

Azure DevOps Timeout Limits Reference

Before applying any fix, confirm which limit applies to your scenario. The following table shows the hard limits and defaults you are working within:

ScenarioDefault timeoutInMinutesMaximum (YAML settable)Hard ceiling
Microsoft-hosted agent — private repo60 minutes60 minutes (free)60 min on free tier. No YAML override changes this.
Microsoft-hosted agent — public repo60 minutes360 minutes (6 hours)360 min regardless of YAML setting
Microsoft-hosted agent — paid parallelism60 minutesNo fixed ceiling in YAMLPractical: set to what your job needs
Self-hosted agent60 minutes (YAML default if not set)No platform ceiling — you control itNone — set timeoutInMinutes: 0 for unlimited
Azure Pipelines default (no YAML override)3,600 minutes (60 hours)Settable per jobThis is why long jobs sometimes appear to "hang" — they are waiting for the 60-hour default to expire
The 5 Fixes
Fix 1Microsoft-Hosted Agent 60-Minute Hard Limit
Most CommonMicrosoft-Hosted

What you see:

Azure DevOps error message##[error]The job running on agent Hosted Agent has exceeded the maximum execution time of 60 minutes.
##[error]Finishing: Finalize Job
##[section]Finishing: Agent job 1
Result: Failed
Duration: 00:60:01

Root cause: Microsoft-hosted agents on the free tier have a hard 60-minute per-job limit for private repositories. This is a platform constraint — not a configuration issue. Setting timeoutInMinutes: 120 in your YAML does nothing on the free tier. The job terminates at exactly 60:00 regardless of what YAML says. The limit is 360 minutes (6 hours) for public repositories and has no fixed ceiling on paid parallelism plans.

How to confirm it is this problem: the job ran successfully to minute 60:00 and then terminated mid-execution. The total duration shown in the pipeline run summary is exactly 60 minutes. There is no network error, no agent crash — just the exceeded execution time message.

1

Confirm the exact duration in the pipeline run summary

Open the failed run → click the job → check Duration at the top of the job page. If it shows 00:60:00 or 00:60:01, it is the hard limit. If it shows a different duration, you have a different problem.

2

Option A: Split the job into multiple smaller jobs

The most architecturally sound fix. Break your pipeline into jobs that each complete in under 50 minutes (leave headroom). Jobs run on separate agent instances and each gets its own 60-minute window. Use dependsOn to chain them in sequence. See YAML below.

3

Option B: Purchase paid parallelism for the organisation

In Azure DevOps → Organization Settings → Billing → Add Microsoft-hosted parallel jobs. Cost: $40/month per parallel job (as of 2026). Paid plans have no per-job execution time ceiling. One parallel job purchase unlocks unlimited per-job duration on Microsoft-hosted agents for all pipelines in the organization.

4

Option C: Migrate the long-running job to a self-hosted agent

Self-hosted agents have no platform-imposed execution time ceiling. Create an Azure VM (Standard B2s is sufficient for most build workloads), install the Azure Pipelines agent, and target the job to your self-hosted pool. The job duration is then limited only by your YAML timeoutInMinutes setting — set it to 0 for unlimited.

5

Option D: Optimise the job to complete in under 55 minutes

Profile which tasks take the longest using the pipeline run summary timeline view. Common optimizations: enable NuGet/npm/pip caching to avoid re-downloading packages on every run, parallelize independent tasks using the matrix strategy, and remove redundant test runs or artifact publishing steps that are not needed for the specific pipeline stage.

YAML — Split one long job into two chained jobs, each under 60 minutes# Before: One job that takes 90 minutes — fails at 60:00
# jobs:
# - job: BuildAndTest
# steps:
# - task: ... # 35 mins
# - task: ... # 30 mins
# - task: ... # 25 mins ← KILLS HERE at 60:00

# After: Two jobs chained with dependsOn
jobs:
- job: Build
timeoutInMinutes: 55 # Explicit — leaves 5 min headroom
pool:
vmImage: 'ubuntu-latest'
steps:
- task: ... # 35 mins
- task: ... # 15 mins — total: ~50 mins ✓
# Publish artifact so next job can pick it up
- task: PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
artifact: 'drop'

- job: Test
dependsOn: Build # Waits for Build to succeed
timeoutInMinutes: 55
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadPipelineArtifact@2
inputs: { artifact: 'drop' }
- task: ... # 25 mins — total: ~25 mins ✓
💡 Pipeline Caching: Fastest Way to Cut Job Duration

Enable the Cache@2 task to cache npm, pip, Maven, NuGet, or Gradle dependencies between runs. A cache hit on a large dependency tree saves 5–20 minutes per job — often the difference between a job that fits in 60 minutes and one that does not.

Example: - task: Cache@2key: '"npm" | "$(Agent.OS)" | package-lock.json'path: $(npm_config_cache)

Fix 2Self-Hosted Agent Heartbeat Loss
Self-Hosted AgentsSilent Failure

What you see:

Azure DevOps error message##[error]We stopped hearing from agent Azure Pipelines. Verify the agent machine is
running and has a healthy network connection. Anything that terminates an agent
process, starves it for CPU, or blocks its network access can cause this error.
Rebuilding the agent pool or restarting the agent service may resolve this issue.
Duration: 01:20:35

Root cause: The agent process lost communication with Azure DevOps while a job was running. The agent sends regular heartbeat messages to Azure DevOps. When those stop for longer than the heartbeat timeout period, Azure DevOps marks the agent as unresponsive and fails the job. This is a symptom — the underlying cause is what you need to fix. Common causes: the agent VM ran out of memory or disk space during the job, a long-running task consumed all CPU and starved the agent process, the network connection between the agent VM and Azure DevOps was interrupted, or the agent process was killed by an OOM killer or OS process manager.

1

Check agent VM resource consumption at the time of failure

SSH into the agent VM and check: free -h (memory), df -h (disk), top (CPU). If memory is near zero or disk is at 100%, the agent was resource-starved. Check dmesg | grep -i "killed" for OOM killer events.

2

Review the agent's internal diagnostic logs

Self-hosted agents store detailed internal logs in the _diag folder at the agent installation root (e.g., /home/azureagent/_diag/ or C:\agent\_diag\). The most recent Agent_*.log file contains the last actions the agent performed before losing contact. Look for the last successful log entry to identify exactly where execution stopped.

3

Fix: Memory — increase VM size or reduce job memory footprint

If OOM killed: resize the agent VM to a larger SKU (e.g., Standard_D4s_v5 instead of Standard_B2s), or split memory-intensive steps into separate jobs. For Docker builds, set --memory limits on containers to prevent them from consuming all available agent memory.

4

Fix: Disk — add a workspace clean step and increase agent disk

Add a workspace clean directive to the job: workspace: clean: all under the job definition. This cleans old build artifacts before each run. Also configure Docker to prune unused images: docker system prune -f as a pre-job step to prevent Docker layer accumulation from filling the disk.

5

Fix: Network — verify agent connectivity to Azure DevOps endpoints

The agent requires outbound HTTPS (port 443) to specific Azure DevOps endpoints. Test connectivity: curl -v https://dev.azure.com and curl -v https://vsrm.dev.azure.com from the agent VM. If these fail, the firewall or NSG is blocking agent communication. See Fix 4 for proxy/firewall configuration.

6

Fix: Set cancelTimeoutInMinutes to give long tasks time to report

Add cancelTimeoutInMinutes: 5 to your job definition. This gives the agent 5 minutes to complete any cleanup tasks and report final status when the job is cancelled due to timeout — giving you more diagnostic information in the logs before the run ends.

YAML — Self-hosted job with workspace clean, disk prune, and cancelTimeoutjobs:
- job: Build
timeoutInMinutes: 180 # 3 hours — set to what you actually need
cancelTimeoutInMinutes: 5 # Give agent time to report before kill
pool:
name: 'Myself-hostedPool'
workspace:
clean: all # Clean workspace before every job run
steps:
- script: |
# Prune Docker to prevent disk exhaustion
docker system prune -f --volumes
df -h # Log disk state before build begins
free -h # Log memory state
displayName: 'Pre-build: clean Docker + log resources'
Bash — Diagnose agent heartbeat loss from _diag logs# On the agent VM — find the most recent diagnostic log
ls -lt /home/azureagent/_diag/Agent_*.log | head -5

# Read the last 100 lines of the most recent log
tail -100 /home/azureagent/_diag/Agent_20260101-120000-utc.log

# Check for OOM kill events
dmesg | grep -i "out of memory\|Killed process"

# Check current disk usage
df -h

# Check memory available
free -h

# Restart the agent service (Linux)
sudo systemctl restart vsts.agent.*
sudo systemctl status vsts.agent.*
Figure 2 — Parallelism quota: why jobs queue forever even when agents appear idle
Organisation Parallelism Quota: 1 Free Parallel Job (Free Tier) or N Purchased JobsParallel Slot 1 — IN USEPipeline: main branch CIAgent: Hosted Ubuntu — running 47 minsJob consuming the only free slotParallel Slot 2 — DOES NOT EXIST(Free tier = 1 slot only)Purchase for $40/month to unlockJobs Waiting in Queue (will wait indefinitely until slot 1 completes)PR #47 build — queued 23 minsHotfix deploy — queued 8 minsFeature branch CI — queued 41 mins+ 4 more queued
On the free tier, only one pipeline job can run at a time across your entire organisation. All other jobs queue until the running job completes. This is not a timeout — it looks like one because the job never starts running at all.
Fix 3Pipeline Stuck in Queue — Parallelism Quota Exhausted
Org-Level IssueAll Agent Types

What you see:

Azure DevOps — pipeline run summary##[error]No hosted parallelism has been purchased or granted.
To request a free parallelism grant, please fill out the following form: https://aka.ms/azpipelines-parallelism-request

— OR —

Status: Queued
Duration: 01:23:47 ← (never moved past Queued state)

Root cause: Azure DevOps limits the number of pipeline jobs that can run simultaneously based on your organisation's parallelism quota. On the free tier, you have exactly one parallel job slot. If that slot is occupied, every other pipeline queues indefinitely. This manifests as a "timeout" when teams misread a never-started job as one that timed out during execution. New Azure DevOps organisations no longer receive a free parallelism grant automatically — Microsoft disabled automatic free grants to prevent abuse. You must either request a grant or purchase parallelism.

1

Confirm this is a parallelism issue — not a timeout

Open the queued pipeline run. If the status is "Queued" and the job never moved to "Running" — even after 10+ minutes — it is a parallelism problem. In Organisation Settings → Pipelines → Parallel jobs, check "Used" vs "Allocated". If Used = Allocated, every new job will queue.

2

Request a free parallelism grant (for legitimate projects)

Submit the Microsoft form at aka.ms/azpipelines-parallelism-request. Approval typically takes 2–3 business days. This is a one-time request and grants one free parallel job for public or private projects.

3

Purchase additional parallel jobs (for production teams)

Organisation Settings → Billing → Add Microsoft-hosted parallel jobs ($40/month each) or Self-hosted parallel jobs ($15/month each). Purchasing unlocks both the additional slot and removes the per-job 60-minute ceiling on Microsoft-hosted agents.

4

Immediate workaround: migrate to a self-hosted agent

Self-hosted agent slots are free (one parallel job per agent machine). Install an agent on any VM or machine you control and target your pipeline to the self-hosted pool. This bypasses the Microsoft-hosted parallelism limit entirely and provides unlimited parallel slots (one per agent machine).

5

Reduce wasted parallelism: cancel stale PR builds automatically

Old PR builds consuming parallelism slots while the PR has been updated are wasted slots. Configure pipeline triggers to automatically cancel in-progress runs when a new commit is pushed to the same PR. Add to your YAML trigger configuration: pr: autoCancel: true.

YAML — Auto-cancel stale PR builds to free parallelism slotstrigger:
branches:
include: [main, develop]

pr:
branches:
include: ['*']
autoCancel: true # Cancel in-progress run when new commit pushed to same PR

jobs:
- job: CI
timeoutInMinutes: 45
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo "Build $(Build.BuildNumber)"
Fix 4Proxy or Firewall Blocking Task Network Calls
Self-HostedCorporate Network

What you see:

Azure DevOps task log — verbose mode##[error]Error: connect ECONNREFUSED 13.107.42.18:443
##[error]Error: connect ETIMEDOUT dev.azure.com:443
##[error]HTTP 407 Proxy Authentication Required
Task: NuGetCommand — hung for 14 minutes with no output
##[error]The process '/usr/bin/dotnet' failed with exit code 1

Root cause: Self-hosted agents behind corporate firewalls or HTTP proxies fail in ways that produce no useful error messages by default. The agent connects to Azure DevOps successfully at startup (polling works), but individual tasks — NuGet restore, npm install, Docker push, Helm deploy — make their own network calls. These task-level calls do not automatically inherit the agent's proxy configuration. The task hangs waiting for a network response that never comes, then eventually hits the job timeout.

1

Enable verbose logging to confirm network calls are hanging

In the pipeline run, set the variable System.Debug = true (see the Verbose Logging section at the end of this guide). With verbose logging, the task log will show each network call — making it clear that a specific HTTPS call is hanging or returning a non-200 response.

2

Test network connectivity from the agent VM

SSH into the agent machine and test the specific endpoints each task needs: curl -v https://api.nuget.org/v3/index.json, curl -v https://registry.npmjs.org, curl -v https://dev.azure.com/YOUR-ORG. A connection refused or timeout confirms the firewall is blocking it.

3

Configure the agent proxy settings

If a proxy is required, configure it in the agent's .proxyconfig file (see CLI command below). This tells the agent process to route traffic through the proxy. But note: this only affects the agent heartbeat and job coordination traffic — not necessarily task-level calls.

4

Set proxy environment variables for task-level calls

Add the following pipeline variables so individual tasks can discover and use the proxy: http_proxy, https_proxy, no_proxy. These are standard environment variables respected by curl, npm, pip, git, Docker, and most CLI tools.

5

Work with your network team to allow Azure DevOps endpoints

The Azure Pipelines agent requires outbound HTTPS to a specific set of endpoints. Get your network team to allow dev.azure.com, *.dev.azure.com, *.vsassets.io, *.vsrm.visualstudio.com, and *.pkgs.visualstudio.com on port 443. The full list is in the Microsoft documentation at aka.ms/azpipelines-agents-firewall.

Bash — Configure agent proxy settings and test connectivity# Configure the agent's proxy (run from agent installation directory)
# Linux / macOS
./config.sh --proxyurl http://proxy.company.com:8080 \
    --proxyusername proxyuser \
    --proxypassword yourpassword

# Or edit the .proxyconfig file directly
echo "http://proxy.company.com:8080" > /home/azureagent/agent/.proxyconfig

# List endpoints that must be accessible from the agent VM
ENDPOINTS=(
  "dev.azure.com"
  "vsrm.dev.azure.com"
  "vsblob.blob.core.windows.net"
  "*.pkgs.visualstudio.com"
)
for ep in "${ENDPOINTS[@]}"; do
  echo -n "Testing $ep: "
  curl -s --max-time 5 "https://$ep" > /dev/null && echo "OK" || echo "FAILED"
done
YAML — Set proxy environment variables for all pipeline tasksvariables:
http_proxy: "http://proxy.company.com:8080"
https_proxy: "http://proxy.company.com:8080"
no_proxy: "localhost,127.0.0.1,.internal.company.com"

jobs:
- job: Build
pool:
name: 'SelfHostedPool'
steps:
- script: |
# Verify proxy is working before main build tasks
curl -v --proxy-insecure https://dev.azure.com/$(System.TeamFoundationCollectionUri)
displayName: 'Connectivity check'
Fix 5Expired Service Connection Causing Silent Task Hang
Deploy PipelinesCascading Failure

What you see:

Azure DevOps task log — deploy stage##[section]Starting: AzureWebApp@1
Preparing deployment...
...
[No further output for 18 minutes]
...
##[error]The job running on agent has exceeded the maximum execution time of 60 minutes.

— In verbose logs —
##[debug]Response code: 401 Unauthorized
##[debug]Token has expired. Token expiry: 2026-03-15T00:00:00Z

Root cause: An Azure service connection uses a service principal with a client secret (or a certificate) to authenticate to the Azure subscription. Service principal secrets have an expiry date — typically 1 or 2 years. When the secret expires, deploy tasks that call the Azure Resource Manager API receive a 401 Unauthorized response. The task does not fail fast — it retries, waits, or hangs silently. The result is a task that appears to be "doing something" for many minutes before finally timing out the entire job. The timeout is a symptom. The expired credential is the actual problem.

1

Confirm the service connection is expired

Project Settings → Service connections → find the connection used by the failing task. Click "Edit" and then "Verify". If verification fails with an authentication error, the credential has expired. Also check the expiry date in Entra ID: search for the service principal name → Certificates & Secrets → verify the secret expiry date.

2

Rotate the service principal secret

In Microsoft Entra ID → App registrations → find the service principal → Certificates & secrets → New client secret → set expiry → copy the new value immediately (it only shows once). Then update the service connection in Azure DevOps: Project Settings → Service connections → Edit → update the service principal key field with the new secret → Verify and Save.

3

Re-run the failed pipeline to confirm the fix

After updating the service connection, re-run the failed pipeline. The deploy task should now authenticate successfully within the first 30 seconds. If it still hangs, the service connection may be pointing to the wrong subscription, or the service principal may be missing RBAC roles on the target resource group.

4

Long-term fix: migrate to Workload Identity Federation

Azure DevOps now supports Workload Identity Federation (OIDC) for service connections — this eliminates the need for a service principal secret entirely. The service connection federates with Azure DevOps's OIDC issuer; no secret is stored, no secret expires. Convert the service connection in Project Settings → Service connections → Edit → convert to Workload Identity Federation. Requires Azure DevOps 2022+ and subscription with Microsoft Entra integration.

5

Prevent future expiry: set a calendar reminder and use Key Vault rotation

Service principal secrets default to 1-2 year expiry but give no automatic notification when they expire. Set a recurring calendar reminder 60 days before expiry. For automated management, store the secret in Azure Key Vault with rotation enabled and configure a Key Vault event to trigger a Logic App or Azure Automation runbook that updates the service connection programmatically.

Azure CLI — Check service principal secret expiry and rotate# List all service principals and their secret expiry dates
az ad app list --display-name "YOUR-SERVICE-CONNECTION-NAME" \
    --query "[].{AppId:appId, Name:displayName}" -o table

# Check credential expiry for a specific app
az ad app credential list \
    --id YOUR-APP-ID \
    --query "[].{End:endDateTime, Description:customKeyIdentifier}" -o table

# Add a new client secret (valid 1 year)
az ad app credential reset \
    --id YOUR-APP-ID \
    --append \
    --years 1 \
    --display-name "ADO-Service-Connection-$(date +%Y-%m)" \
    --query "{ clientId: appId, clientSecret: password }" -o json

# Copy the clientSecret value and update in Azure DevOps
# Project Settings → Service Connections → Edit → key field
⚠ Workload Identity Federation: The Permanent Fix

Converting to Workload Identity Federation (OIDC-based) service connections eliminates service principal secrets entirely. No secret means no expiry. No expiry means no surprise pipeline failures at 2am six months after someone created a service connection and forgot to document when the secret expires. This migration takes approximately 10 minutes per service connection and is the long-term answer to Fix 5.

Prevention: YAML Timeout Defaults and Monitoring Every Pipeline Should Have

The five fixes above address failures after they happen. These controls prevent them from occurring silently and reduce the time-to-diagnosis when they do.

1. Always Set Explicit timeoutInMinutes on Every Job

The Azure Pipelines default job timeout is 3,600 minutes — 60 hours. A hung job on a self-hosted agent will wait 60 hours before failing if you have not set an explicit timeout. Every job in every pipeline should have an explicit timeoutInMinutes set to the maximum legitimate time it should take, plus a 20% buffer.

YAML — Production pipeline template with all timeout and monitoring controls# Complete pipeline with explicit timeouts, retry, and monitoring
name: $(Build.DefinitionName)-$(Build.BuildNumber)

trigger:
branches: { include: [main, release/*] }
pr: { autoCancel: true } # Cancel stale PR builds to free slots

variables:
TIMEOUT_BUILD: 45 # Max legitimate build time + 20% buffer
TIMEOUT_TEST: 30
TIMEOUT_DEPLOY: 20

stages:
- stage: Build
jobs:
- job: BuildApp
timeoutInMinutes: $(TIMEOUT_BUILD)
cancelTimeoutInMinutes: 5 # Give agent time to report before kill
pool: { vmImage: 'ubuntu-latest' }
retryCountOnTaskFailure: 1 # Retry transient failures once
steps:
- # Cache dependencies to cut job time
- task: Cache@2
inputs:
key: '"npm" | "$(Agent.OS)" | package-lock.json'
path: $(npm_config_cache)
restoreKeys: 'npm | "$(Agent.OS)"'
- script: npm ci
- script: npm run build
- task: PublishPipelineArtifact@1
inputs: { targetPath: dist, artifact: drop }

- stage: Deploy
dependsOn: Build
condition: succeeded()
jobs:
- deployment: DeployProd
timeoutInMinutes: $(TIMEOUT_DEPLOY)
environment: production
strategy: { runOnce: { deploy: { steps: [
- task: AzureWebApp@1
inputs:
azureSubscription: 'MyServiceConnection' # Must be verified before run
appName: my-webapp
]}}}

Enabling Verbose Diagnostic Logs in Azure Pipelines

When a pipeline fails with a timeout and the logs show nothing useful, the first step is always enabling verbose logging. Verbose mode logs every API call, every network request, every internal agent action — exposing what was happening in the seconds before the failure.

1

Method 1 — Add System.Debug variable to the pipeline run

In the pipeline, click "Run pipeline" → Variables → Add variable → Name: System.Debug → Value: true. This enables verbose logging for that single run without modifying your YAML. Use this for one-off diagnosis.

2

Method 2 — Add System.Debug to YAML variables (temporarily)

Add System.Debug: true under the variables section of your YAML. Remove it after diagnosis — verbose logs are significantly larger and slower to load.

3

Read the verbose logs: look for the last successful action

In the verbose task log, every line is timestamped. Find the last timestamp before the silence began. That timestamp tells you exactly which operation the agent was waiting on when it stopped responding. A gap of several minutes between log entries is the signal — identify what was called immediately before the gap.

4

Download raw logs for large failures

In the pipeline run → top-right menu → "Download logs". The downloaded ZIP contains raw log files for every task. The agent_diagnostic.log file inside the ZIP contains agent-level events that do not appear in the pipeline UI — particularly useful for heartbeat loss and proxy failures.

Quick Reference: Match Your Error to the Fix

"The job has exceeded the maximum execution time of 60 minutes" on a vmImage job = Fix 1. The Microsoft-hosted 60-minute cap is a hard platform limit. Setting timeoutInMinutes higher does nothing on the free tier. Split the job, optimise it, or buy parallelism.
"We stopped hearing from agent" = Fix 2. The self-hosted agent lost contact. Check _diag/Agent_*.log on the agent VM. Check memory (free -h), disk (df -h), and OOM events (dmesg | grep Killed). Add workspace: clean: all and cancelTimeoutInMinutes: 5.
Job stuck in "Queued" state, never moves to "Running" = Fix 3. This is a parallelism limit, not a timeout. Check Organisation Settings → Parallel jobs. Purchase a slot or migrate to self-hosted. Enable pr: autoCancel: true to free slots from stale PR builds.
ECONNREFUSED, ETIMEDOUT, 407, or tasks hanging silently on self-hosted agents = Fix 4. A firewall or proxy is blocking task-level network calls. Enable verbose logging, test connectivity from the agent VM with curl, and set http_proxy and https_proxy as pipeline variables.
Deploy task hangs for 10–20 minutes then job times out — visible 401 in verbose logs = Fix 5. An expired service principal secret. Verify the service connection in Project Settings, rotate the secret in Entra ID, update the service connection, and migrate to Workload Identity Federation to prevent this permanently.
Every pipeline should have explicit timeoutInMinutes set. The default is 3,600 minutes — 60 hours. A hung self-hosted job will block that agent for 60 hours if you do not set a realistic timeout. Set it to the legitimate maximum plus 20% buffer on every job, every pipeline.
Verbose logging (System.Debug: true) is your first diagnostic tool. Pipeline timeouts rarely leave useful error messages by default. Verbose logging exposes every API call and network request — the last entry before the silence tells you exactly what the agent was waiting on when it stopped responding.

Frequently Asked Questions

I set timeoutInMinutes: 120 in my YAML but the job still fails at 60 minutes. Why?
You are on the Microsoft-hosted free tier with a private repository. The 60-minute hard ceiling applies at the platform level regardless of your YAML setting. Azure Pipelines ignores your timeoutInMinutes value when it exceeds the platform limit. Your options are: split the job into smaller pieces, purchase paid parallelism (which removes the ceiling), migrate the long-running job to a self-hosted agent (no ceiling), or convert your repository to public (6-hour ceiling). This is the most frequently misunderstood Azure DevOps timeout behavior.
My self-hosted agent shows as "Online" in the portal, but jobs still get stuck queued. What is wrong?
An agent showing "Online" in the pool means the agent service is running and polling Azure DevOps for jobs. It does not mean the agent is available to pick up new jobs. Check three things: (1) the agent may be already running a job — look at the agent pool's "Jobs" tab to see current assignments; (2) the pipeline may not be authorized to use the self-hosted pool — go to the pool → Security and confirm your pipeline is in the authorized list; (3) the pipeline's demands (the demands key in YAML) may specify capabilities the agent does not have — the agent will not be assigned the job even if it is online.
How do I configure pipeline timeout for deployment jobs specifically vs build jobs?
Deployment jobs (using the deployment keyword) support timeoutInMinutes at the job level, just like regular jobs. Set it under the deployment job definition. Deployment jobs also support cancelTimeoutInMinutes — particularly important because deployment jobs perform environment teardown on cancel, and you want to give them time to complete that cleanup before the agent is killed. A reasonable default is timeoutInMinutes: 30 and cancelTimeoutInMinutes: 10 for production deployments.
How do I get notified immediately when a pipeline times out rather than finding out hours later?
Go to Project Settings → Notifications → New subscription → Build category → "A build fails". Set the delivery method to email or Teams. For instant Teams notifications, create a pipeline webhook integration: project → Pipelines → the specific pipeline → Edit → Notifications (top right). For Azure Monitor-level alerting, use the Azure DevOps REST API with a Logic App to query run status and post to Teams/Slack when a run transitions to Failed. The built-in notification system fires within 2–5 minutes of a run completing, which is the most reliable option for most teams.

Popular posts from this blog

Learn how to use Azure Chaos Studio to simulate data center outages, test Azure OpenAI failover, and validate AI app resiliency using KQL and CLI workflows

Resiliency Testing Chaos Studio Zone Down Azure OpenAI Failover Testing AI Resiliency: Using Azure Chaos Studio to Simulate Data Center Outages on Your LLM Every multi-region Azure OpenAI architecture diagram has a failover arrow drawn on it. Almost none of them have ever actually been triggered. The arrow is a hypothesis, confirmed only by a real outage — unless you deliberately cause a controlled one first, on your own schedule, with a rollback plan, instead of finding out during an incident that the failover you designed never quite worked the way the diagram promised. The failure signature this guide resolves # The gap this article closes — a real architecture review finding: Design doc, page 4: "In the event of a regional outage, Azure Front Door automatically routes traffic to the secondary Azure OpenAI deployment in West Europe, with an expected failover time under 60 seconds." Verification performed to support this claim: NONE. Last time this path was ac...

Improve AI application performance by reducing latency, optimizing embeddings, and lowering cloud inference costs

Performance Fix Foundry Local 1.2 Linux ARM64 Embeddings Offline ASR The Edge Latency Drop: Fixing Latency Spikes by Offloading Embeddings to Foundry Local 1.2 You are paying a full cloud round trip — network, TLS, queue, throttle risk — to turn a twelve-word search query into a vector. That is the most expensive way possible to do one of the cheapest computations in your stack. Foundry Local 1.2 now runs on Linux ARM64, which means embeddings and speech recognition can happen on a Raspberry Pi, a Jetson, or a Graviton instance — offline, unmetered, and in single-digit milliseconds. The failure signature this guide resolves # Application Insights — the embedding call, not the LLM, is your tail latency: name p50 p95 p99 calls/day POST /embeddings (cloud) 89 ms 412 ms 3,847 ms 1,240,000 POST /chat/completions (cloud) 940 ms 1,720 ms 2,910 ms 38,000 ^^^^^^^^ ...

Learn how to select Azure Files and Blob storage tiers, avoid early deletion fees, model costs, and automate lifecycle management for large file migrations.

Choosing the Right Azure Storage Tier for Large File Migrations The complete decision framework for storage tier selection during large file migrations — Azure Files tiers, Blob tiers, cost modelling, early deletion traps, lifecycle automation, and the 2026 changes that affect every migration running today. By Francis Avorgbedor | Azure Engineer  ·  July 14, 2026  ·  18 min read  ·  Storage Tiers · Cost Optimisation · Migration FA Francis Avorgbedor Azure Engineer  ·  SEVENAI  ·  Azure Field Notes 9 Distinct Azure storage tiers across Files and Blob — most engineers know only 3 15hrs Archive tier rehydration time at standard priority — the delay teams forget to plan for 128KB Minimum billable object size for Cool/Cold/Archive from July 2026 — a 32× trap for small files 70% How much retrieval and transaction fees add to a theoretical Archive storage bill The most expensive mistake I see on large Azure file migrations is not choosing the w...

Find the hidden Windows 10 system files consuming up to 500GB, including hibernation, shadow copies, backups, and WinSxS, with safe cleanup steps.

  The 500GB System File That Eats Your Hard Drive Something on your Windows 10 drive is consuming hundreds of gigabytes and the normal tools cannot find it. This guide identifies every known culprit — from hibernation files and shadow copies to runaway backups and the Windows component store — and tells you exactly what is safe to delete, what to leave alone, and what the commands actually do.

Learn safe methods to reset Azure virtual machines using managed disks while preserving critical workloads

How to Reset an Azure Virtual Machine to Factory Settings Using a Managed Disk Azure does not have a single "factory reset" button. What it does have is something better: the OS Disk Swap — a method that swaps out the corrupted or misconfigured OS disk for a clean Windows Server managed disk without deleting the VM, its NICs, its IP addresses, or any attached data disks. Here is how it works, when to use it, and the exact steps to execute it safely. FA Francis Avorgbedor Azure Engineer July 16, 2026 15 min read Azure VMs · Windows Server · Real-World Fix 3 Methods to achieve a clean Windows Server installation on an existing Azure VM ~15min Typical OS Disk Swap duration — VM retains its NICs, IPs, and data disks throughout 0 Data disks affected by an OS Disk Swap — data disks remain attached and untouched 1 Snapshot of the original OS disk you must take before starting — no exceptions Introduction Why Azure Does Not Have a Simple Factory Reset — and What to Do Instead On a ph...

Determine Windows 11 compatibility, upgrade requirements, costs, and performance expectations on older hardware

Can I Update My Old Computer to Windows 11 — and How Much Will It Cost? Your i7, 16GB RAM, 512GB SSD machine is powerful enough to run Windows 11 comfortably. The TPM 2.0 and Secure Boot wall is a security checkbox, not a performance ceiling. Here are two proven ways to get past it, what each one costs, and what you are trading away by doing so. $0 Cost of the Windows 11 licence if your existing Windows 10 is genuine — the upgrade remains free in 2026 2 Proven methods to bypass TPM 2.0 and Secure Boot — Rufus (easy) and Registry edit (manual) 25H2 Current Windows 11 version — all known bypass methods tested and confirmed working as of July 2026 Oct 2025 Windows 10 end of life — no more security updates. Staying on Windows 10 now carries real risk. First — Check Your BIOS Before Anything Else You Might Not Actually Need a Bypass Before running any bypass, open your BIOS and look at two settings. Many computers that fail the Windows 11 compatibility check have TPM 2.0 present in the hard...

Solve common AKS issues with practical troubleshooting techniques for networking, scaling, upgrades, and workloads

Troubleshooting Guide AKS Kubernetes Real Solutions kubectl Azure Kubernetes Service (AKS) Troubleshooting Guide: Real Solutions to Common Problems CrashLoopBackOff at 2am. Pods stuck Pending with no obvious cause. Nodes going NotReady mid-deployment. DNS resolution silently failing in production. Every AKS engineer encounters these — the difference between engineers who panic and engineers who stay calm is knowing the exact sequence of diagnostic commands to run. This guide gives you that sequence, the root cause analysis for each failure mode, and the fix. 3 commands 90% of AKS problems are diagnosed with the same three kubectl commands: describe pod, logs --previous, and get events — in that order, every time Exit 137 The exit code that tells you everything: container killed by SIGKILL — either the Linux OOM killer (memory limit exceeded) or kubelet after grace period expired 5 min The CrashLoopBackOff ceiling: Kubernetes applies exponential backoff (10s → 20s → 40s → 80s → 160s → 3...

Step-by-step guide to deploying scalable AI chatbots on Azure with OpenAI and App Service

Step-by-Step Guide Azure OpenAI App Service Production Python How to Deploy an AI Chatbot on Azure Using Azure OpenAI and App Service From zero to a production-grade AI chatbot: provision Azure OpenAI, write a streaming Flask API backend, deploy it on Azure App Service with Managed Identity, wire in conversation history and content safety, and instrument it with Application Insights — all with complete code and Terraform IaC. No API keys in environment variables. No hardcoded secrets. No half-finished PoC patterns. 7 phases This guide covers the full deployment lifecycle: architecture design → resource provisioning → backend code → App Service deployment → streaming → security → monitoring Zero keys The chatbot authenticates to Azure OpenAI using Managed Identity and DefaultAzureCredential — no API keys stored in environment variables, Key Vault, or code SSE Server-Sent Events stream GPT tokens to the browser as they generate — the same token-by-token typing effect users expect from pr...