Learn how Azure OpenAI works, including GPT models, pricing, deployment options, enterprise security, and AI application development on Azure.
What Is Azure OpenAI and How Does It Work?
Microsoft's managed cloud service for accessing the world's most capable AI models — GPT-5, GPT-4.1, o-series reasoning models, Sora, DALL-E, Whisper — inside Azure's enterprise security boundary. This guide explains the architecture, model catalogue, deployment types, pricing, and how to start building.
What Is Azure OpenAI?
Azure OpenAI is Microsoft's managed cloud service that gives organisations API-level access to OpenAI's most advanced AI models — including GPT-5, GPT-4.1, o-series reasoning models, DALL-E, Whisper, and Sora — while keeping all data, computation, and traffic inside Microsoft Azure's security and compliance boundary. It is the enterprise version of OpenAI's technology, delivered through Azure's global infrastructure with the governance controls, private networking, and regulatory certifications that large organizations require.
The service is the product of a deep partnership between Microsoft and OpenAI that began in 2019. Microsoft first integrated OpenAI models into Azure in 2021, made the service generally available in January 2023, and by 2026 had evolved it into the Azure AI Foundry platform — a unified environment that brings together Azure OpenAI models alongside over 11,000 other models from providers including Anthropic (Claude), Meta (Llama 4), Mistral, Cohere, DeepSeek, and xAI (Grok). Azure OpenAI is the anchor of that ecosystem.
The fundamental value proposition is straightforward: you get access to exactly the same underlying AI models as developers using OpenAI's direct API, but with your data never leaving Azure's infrastructure, full Microsoft Entra ID authentication, VNET integration, private endpoints, and the same compliance certifications that cover all other Azure services your organization already uses.
Azure OpenAI vs OpenAI API — What Is the Difference?
The models are the same. The difference is entirely in infrastructure, data handling, compliance, and integration. Here is the complete comparison:
| Factor | OpenAI API (direct) | Azure OpenAI |
|---|---|---|
| Underlying models | GPT-5, GPT-4.1, o-series, DALL-E, Whisper, Sora | Same models — API-compatible with OpenAI API |
| Data handling | Data processed on OpenAI infrastructure. Not used for training by default, but subject to OpenAI's data policies | Data stays within your Azure tenant and region. Microsoft commits: data is not used to train models, is not accessible to OpenAI |
| Authentication | API key (bearer token) | Microsoft Entra ID (OAuth 2.0) plus API key option |
| Private networking | Public internet only | Azure VNET integration, private endpoints, no public internet required |
| Data residency | OpenAI data centres (US-based primarily) | Your chosen Azure region, plus Data Zone and Global options. EU Data Boundary available |
| Compliance certifications | SOC 2, ISO 27001 | 50+ certifications including HIPAA, FedRAMP High, ISO 27001, SOC 2 Type II, PCI DSS, GDPR |
| Content filtering | OpenAI built-in moderation | Azure content filters (configurable), plus Prompt Shields, groundedness detection, custom blocklists |
| Deployment control | Model versions managed by OpenAI — auto-updated | You control exactly which model version is in production. Updates are explicit, not automatic |
| Azure integration | Manual — requires network egress from Azure | Native — Cosmos DB, Azure AI Search, Azure Monitor, Azure Fabric, Key Vault, Managed Identity |
| SLA | Uptime SLA via OpenAI | Microsoft Azure SLA (99.9%+), including 99% latency SLA on token generation for PTU deployments |
| Support | OpenAI support channels | Microsoft Azure Support, Premier Support, partner support ecosystem |
| Best for | Startups, individual developers, rapid prototyping without compliance requirements | Enterprise, regulated industries, organisations already on Azure, data residency requirements |
How Azure OpenAI Works — The Architecture
Understanding how Azure OpenAI actually processes a request is essential for building reliable production applications. The journey from your code to a model response and back passes through multiple Azure-managed layers, all within the Azure security boundary.
Key architectural concepts
Azure AI Foundry resource — the top-level Azure resource that contains your Azure OpenAI deployments. Previously called an "Azure OpenAI resource", it now lives within the Azure AI Foundry namespace, giving you access to both Azure OpenAI models and other third-party models from the same resource.
Deployment — a named instance of a specific model (e.g. my-gpt5-prod pointing to the gpt-5 model at a specific version). You create deployments within your AI Foundry resource, choose a deployment type (Standard, Provisioned, or Batch), and set a quota. Multiple deployments of the same or different models can exist in the same resource. Your application always calls a specific deployment by name, not a model directly.
Endpoints and API keys — each Azure AI Foundry resource has a unique HTTPS endpoint (e.g. https://your-resource.openai.azure.com/). Your application authenticates either with an API key or with an Entra ID bearer token and sends requests to the deployment's endpoint. The API surface is fully compatible with OpenAI's API — you can use the OpenAI Python SDK, JavaScript SDK, or any other OpenAI-compatible client library against Azure OpenAI by simply changing the base URL and adding an API version parameter.
Content filtering — runs on both inputs and outputs for every request. Azure's built-in content filters detect harmful content categories (hate speech, sexual content, violence, self-harm) and can block or flag them. Additional Responsible AI features include Prompt Shields (detects jailbreak attempts and indirect prompt injection), groundedness detection (identifies hallucinations in RAG responses), and protected material detection. Filter sensitivity is configurable per deployment.
The Full Model Catalogue (2026)
Azure OpenAI provides access to the full breadth of OpenAI's model families, organized by capability. Model availability varies by region — check the Azure AI Foundry portal for current region availability before planning a deployment.
Deployment Types — Standard, Provisioned, Batch, and Global
When you create a model deployment in Azure OpenAI, you choose a deployment type. This controls how traffic is routed, how costs are calculated, and what performance characteristics you can expect. Choosing the right deployment type is one of the most consequential production architecture decisions you will make.
| Deployment Type | How It Works | Pricing Model | Best For | SLA |
|---|---|---|---|---|
| Standard (Pay-As-You-Go) | Shared multi-tenant infrastructure. Traffic load-balanced across Azure capacity. Variable latency depending on demand. | Per-token (input + output). No upfront commitment. | Development, testing, variable workloads, early production | Uptime SLA; no latency guarantee |
| Provisioned (PTU) | Reserved GPU compute allocated exclusively to your deployment. Consistent, low-latency throughput. Charged hourly regardless of usage. | Provisioned Throughput Units (PTUs) — hourly rate per PTU. Monthly and annual reservations available (lower rate). 99% latency SLA on token generation. | High-volume production, latency-sensitive applications, consistent workloads | Uptime + 99% latency SLA |
| Batch | Asynchronous processing of large prompt sets with 24-hour turnaround. Runs on separate quota from Standard. 50% lower cost than Standard. | Per-token at 50% discount vs Standard | Bulk data processing, overnight analytics, large document ingestion, non-real-time classification | 24-hour turnaround commitment |
| Global Standard | Same as Standard but traffic can route to any Azure region with available capacity, not just your configured region. Higher throughput headroom. | Per-token (slightly higher than regional Standard for some models) | Applications needing higher throughput than regional Standard provides. Supports Spillover to prevent throttling. | Uptime SLA |
| Data Zone | Traffic stays within a geographic zone (e.g. EU Data Zone) without pinning to a single region. Higher throughput than single-region Standard. | Per-token; Standard and Provisioned available | EU data residency requirements with higher throughput than single-region deployment | Uptime SLA |
Security, Privacy, and Compliance
The security architecture is one of the primary reasons enterprises choose Azure OpenAI over OpenAI's direct API. The key distinctions from a data governance perspective:
- Your data is not used to train OpenAI models. Microsoft contractually commits that data submitted to Azure OpenAI is not accessible to OpenAI and is not used to improve any OpenAI model. This is a critical difference from many AI services where user data feeds model training.
- Data does not leave Azure. All processing happens within Azure's infrastructure. Traffic does not traverse the public internet if you use private endpoints, and it never touches OpenAI's own servers.
- Private endpoint support. You can configure your Azure OpenAI resource to be accessible only from within your Azure VNET via private endpoints, with no public internet exposure at all.
- Microsoft Entra ID authentication. Every API call can be authenticated with a managed identity or service principal using standard Azure RBAC, removing the need to manage or rotate API keys.
- Customer-managed keys (CMK). Encryption of data at rest using keys stored in Azure Key Vault, under your control.
- Abuse monitoring with opt-out option. By default, Microsoft may monitor prompt and completion content for abuse detection. Enterprises meeting eligibility requirements can apply to opt out of this monitoring through the Azure portal.
Real-World Use Cases by Industry
Azure OpenAI powers production applications across every major industry. These are the most common and highest-value deployment patterns as of 2026:
Pricing — Pay-As-You-Go vs PTUs
Azure OpenAI pricing operates on two fundamentally different models that suit different workload characteristics. Understanding the trade-offs is critical to avoiding unexpected costs.
Standard (Pay-As-You-Go) pricing
You are billed per token — a subword unit of text (roughly 4 characters or ¾ of a word in English). Pricing is separate for input tokens (the prompt you send) and output tokens (the response generated). Prices vary by model and are higher for more capable models. Standard pricing is ideal for variable workloads, development, and workloads where you cannot predict monthly token consumption. There is no upfront commitment and no minimum spend.
Provisioned Throughput Units (PTU) pricing
You purchase a number of Provisioned Throughput Units (PTUs), which reserve a specific amount of model processing capacity exclusively for your workload. You are charged an hourly rate per PTU regardless of how many tokens you actually consume. This model provides predictable billing, eliminates token-rate variability that affects latency in Standard deployments, and includes a 99% latency SLA on token generation. Monthly and annual PTU reservations are available at a discount over hourly pricing. PTU pricing is best for high-volume, consistent workloads where you can predict utilization.
Batch pricing
Batch deployments are billed per-token at 50% of the equivalent Standard rate. The trade-off is a 24-hour turnaround time rather than real-time responses. Ideal for nightly data processing, bulk document analysis, large-scale classification, and any workload that is not latency-sensitive.
How to Get Started in 5 Steps
Getting from zero to a working Azure OpenAI deployment takes about 20 minutes if you already have an Azure subscription.
# Step 2: Create the Azure AI Foundry / Azure OpenAI resource az cognitiveservices account create \
--name my-openai-resource \
--resource-group rg-openai-demo \
--kind OpenAI \
--sku S0 \
--location eastus
# Step 3: Get the endpoint and key az cognitiveservices account show \
--name my-openai-resource --resource-group rg-openai-demo \
--query properties.endpoint -o tsv
az cognitiveservices account keys list \
--name my-openai-resource --resource-group rg-openai-demo
# Step 4: Create a deployment (model instance) az cognitiveservices account deployment create \
--name my-openai-resource \
--resource-group rg-openai-demo \
--deployment-name my-gpt4-1-deployment \
--model-name gpt-4.1 \
--model-version "2025-04-14" \
--model-format OpenAI \
--sku-capacity 10 \
--sku-name Standard
# Step 5: Test with the OpenAI Python SDK (same SDK works for both OpenAI and Azure OpenAI) # pip install openai
client = AzureOpenAI(
azure_endpoint="https://my-openai-resource.openai.azure.com/",
api_key="YOUR_API_KEY", # or use DefaultAzureCredential() for Entra ID auth
api_version="2024-12-01-preview"
)
response = client.chat.completions.create(
model="my-gpt4-1-deployment", # deployment name, not model name
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain Azure OpenAI in one paragraph."}
],
max_tokens=500,
temperature=0.7
)
print(response.choices[0].message.content)
# Use DefaultAzureCredential for Managed Identity (no key management needed):
# from azure.identity import DefaultAzureCredential, get_bearer_token_provider
# token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
# client = AzureOpenAI(azure_ad_token_provider=token_provider, ...)
Fine-Tuning and Customization
Azure OpenAI supports fine-tuning for select models, allowing you to specialize a base model on your own labelled data. Fine-tuned models produce more consistent, on-brand, and domain-accurate outputs without requiring large context windows or lengthy system prompts on every request. This reduces token consumption and improves both response quality and latency for specific tasks.
Fine-tuning is supported for gpt-4o, gpt-4o-mini, and gpt-35-turbo models in Azure OpenAI. The process involves uploading training data in JSONL format, initiating a fine-tuning job in the Azure AI Foundry portal or via API, waiting for training to complete (typically minutes to hours depending on dataset size), and then deploying the resulting custom model as a standard deployment endpoint.
For production fine-tuning workloads, Azure recommends: a minimum of 50–100 high-quality examples; using chat-format JSONL (system, user, assistant message triplets); splitting data into training and validation sets (90/10 split is typical); and iterating on data quality rather than quantity — 200 excellent examples consistently outperform 2,000 mediocre ones.
Key Takeaways
Related FAVRITE Articles
- How to Fix Databricks Claude PERMISSION_DENIED Rate Limit of 0 (TRIAL_VERIFIED Trust Tier)
- Designing a Secure File Storage Architecture in Azure
- How to Configure Conditional Access for Approved Client Apps
- Top 50 Azure Cloud Administrator Interview Questions and Answers