Once you have selected a model from the catalog, the next step is deploying it. This lesson covers deploying Azure OpenAI models, serverless API models, and managed compute models — including pricing tiers, rate limits, content filters, and versioning.
Official Documentation: Create and Deploy an Azure OpenAI Resource
| Deployment Type | Models | Billing | Infrastructure |
|---|---|---|---|
| Azure OpenAI — Global Standard | GPT-4o, GPT-4o-mini, o3, DALL-E, Whisper, TTS, Embeddings | Pay-per-token | Microsoft-managed, global |
| Azure OpenAI — Standard | Same as above | Pay-per-token | Microsoft-managed, regional |
| Azure OpenAI — Provisioned (PTU) | GPT-4o, GPT-4o-mini | Reserved throughput units | Microsoft-managed, reserved capacity |
| Serverless API (MaaS) | Llama, Mistral, Cohere, Phi | Pay-per-token (Marketplace) | Microsoft-managed |
| Managed Compute | Any open-weight model | VM compute hours | Dedicated VM, user-managed |
gpt-4o)gpt-4o-prod)Defaultfrom azure.identity import DefaultAzureCredential
from azure.mgmt.cognitiveservices import CognitiveServicesManagementClient
from azure.mgmt.cognitiveservices.models import Deployment, DeploymentProperties, DeploymentModel
credential = DefaultAzureCredential()
# Initialize the management client
mgmt_client = CognitiveServicesManagementClient(
credential=credential,
subscription_id="your-subscription-id"
)
# Create a deployment
deployment = Deployment(
properties=DeploymentProperties(
model=DeploymentModel(
format="OpenAI",
name="gpt-4o",
version="2024-11-20"
),
version_upgrade_option="OnceCurrentVersionExpired",
current_capacity=30, # TPM in thousands (30 = 30K TPM)
)
)
result = mgmt_client.deployments.begin_create_or_update(
resource_group_name="your-rg",
account_name="your-openai-resource",
deployment_name="gpt-4o-prod",
deployment=deployment
).result()
print(f"Deployment '{result.name}' status: {result.properties.provisioning_state}")
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint="https://your-resource.openai.azure.com/",
api_key="your-api-key", # Or use DefaultAzureCredential
api_version="2024-12-01-preview"
)
response = client.chat.completions.create(
model="gpt-4o-prod", # Your deployment name
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain quantum computing in 3 sentences."}
],
temperature=0.7,
max_tokens=200
)
print(response.choices[0].message.content)
print(f"Tokens used: {response.usage.total_tokens}")
| Feature | Standard | Global Standard |
|---|---|---|
| Billing | Per 1K tokens consumed | Per 1K tokens consumed |
| Capacity | Regional, shared pool | Global, Microsoft-optimized routing |
| Rate Limits | Set via TPM quota | Set via TPM quota |
| Latency | Consistent within region | Optimized globally |
| Best For | Regional data residency | Most production workloads |
Pricing example (GPT-4o, approximate):
| Feature | Details |
|---|---|
| Billing | Hourly rate per PTU reserved |
| Capacity | Guaranteed, dedicated throughput |
| Minimum | Varies by model (typically 50-100 PTU) |
| Commitment | Monthly or yearly reservations for discounts |
| Latency | Predictable, lower p95 latency |
| Best For | High-volume, latency-sensitive production |
When to choose PTU:
Monthly Token Volume Decision:
├── < 10M tokens/month → Global Standard (PAYG)
├── 10M - 100M tokens/month → Global Standard, evaluate PTU
└── > 100M tokens/month → PTU (likely cost-effective)
Serverless API deployments are available for partner models like Llama, Mistral, and Cohere.
Meta-Llama-3.1-70B-Instruct)from azure.ai.ml import MLClient
from azure.ai.ml.entities import MarketplaceSubscription, ServerlessEndpoint
from azure.identity import DefaultAzureCredential
credential = DefaultAzureCredential()
ml_client = MLClient(
credential=credential,
subscription_id="your-subscription-id",
resource_group_name="your-rg",
workspace_name="your-ai-foundry-project"
)
# Step 1: Subscribe to the model (required for Marketplace models)
marketplace_sub = MarketplaceSubscription(
model_id="azureml://registries/azureml-meta/models/Meta-Llama-3.1-70B-Instruct",
name="llama-31-70b-sub"
)
ml_client.marketplace_subscriptions.begin_create_or_update(marketplace_sub).result()
# Step 2: Create the serverless endpoint
endpoint = ServerlessEndpoint(
name="llama-31-70b-endpoint",
model_id="azureml://registries/azureml-meta/models/Meta-Llama-3.1-70B-Instruct"
)
created_endpoint = ml_client.serverless_endpoints.begin_create_or_update(endpoint).result()
print(f"Endpoint URL: {created_endpoint.scoring_uri}")
print(f"Status: {created_endpoint.provisioning_state}")
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential
client = ChatCompletionsClient(
endpoint="https://your-endpoint.region.models.ai.azure.com",
credential=AzureKeyCredential("your-endpoint-key")
)
response = client.complete(
messages=[
SystemMessage(content="You are a helpful coding assistant."),
UserMessage(content="Write a Python function to find prime numbers up to n.")
],
temperature=0.7,
max_tokens=500
)
print(response.choices[0].message.content)
Azure OpenAI uses Tokens Per Minute (TPM) as the primary rate-limiting metric:
| Deployment Type | Rate Limit Unit | How It Works |
|---|---|---|
| Standard / Global | TPM (Tokens Per Minute) | Shared quota pool per subscription per region |
| Provisioned (PTU) | PTU count | Fixed throughput, no token-based rate limit |
| Serverless API | RPM / TPM | Model-specific, set at deployment |
# Check current quota usage via Azure CLI
# az cognitiveservices usage list --location eastus2 --subscription your-sub-id
# Or use the management SDK
from azure.mgmt.cognitiveservices import CognitiveServicesManagementClient
mgmt_client = CognitiveServicesManagementClient(credential, "your-sub-id")
# List current deployments and their capacity
deployments = mgmt_client.deployments.list(
resource_group_name="your-rg",
account_name="your-openai-resource"
)
for d in deployments:
props = d.properties
print(f"{d.name}: {props.model.name} v{props.model.version} "
f"- Capacity: {props.current_capacity} TPM(K)")
import time
from openai import AzureOpenAI, RateLimitError
client = AzureOpenAI(
azure_endpoint="https://your-resource.openai.azure.com/",
api_key="your-api-key",
api_version="2024-12-01-preview"
)
def call_with_retry(messages, max_retries=5):
"""Call Azure OpenAI with exponential backoff on rate limits."""
for attempt in range(max_retries):
try:
return client.chat.completions.create(
model="gpt-4o-prod",
messages=messages,
temperature=0.7
)
except RateLimitError as e:
retry_after = int(e.response.headers.get("Retry-After", 2 ** attempt))
print(f"Rate limited. Retrying in {retry_after}s (attempt {attempt + 1})")
time.sleep(retry_after)
raise Exception("Max retries exceeded")
response = call_with_retry([
{"role": "user", "content": "Hello!"}
])
Azure OpenAI includes built-in content filtering for safety:
| Category | What It Filters |
|---|---|
| Hate | Hate speech, discrimination |
| Sexual | Sexually explicit content |
| Violence | Graphic violence, self-harm |
| Self-Harm | Self-harm instructions or glorification |
| Jailbreak | Prompt injection attempts (input only) |
| Protected Material | Copyrighted text/code (output only) |
safe, low, medium, high — set thresholds per category| Strategy | Behavior |
|---|---|
Specific version (e.g., 2024-11-20) |
Pinned to exact version; you control upgrades |
| Default | Auto-updated by Microsoft on a published schedule |
| Upgrade policy: OnceCurrentVersionExpired | Auto-upgrades only when current version retires |
| Upgrade policy: NoAutoUpgrade | Never auto-upgrades; you must migrate manually |
# Pin to a specific version for production stability
deployment = Deployment(
properties=DeploymentProperties(
model=DeploymentModel(
format="OpenAI",
name="gpt-4o",
version="2024-11-20" # Pin the version
),
version_upgrade_option="NoAutoUpgrade", # Control upgrades
)
)
💡 Tip: Always pin model versions in production. Use a staging deployment on
Defaultto test new versions before rolling them out.
💡 Start with Global Standard for new workloads. It offers the best availability and simplest setup. Move to Provisioned only when you have predictable high-volume traffic.
💡 Use deployment names as an abstraction layer. Your application code references the deployment name, not the model name. This lets you swap models or versions without changing code.
💡 Set up multiple deployments for the same model — one for production (pinned version, higher TPM) and one for development (latest version, lower TPM).
💡 Monitor 429 errors closely. A spike in rate-limit errors means you need more TPM quota or should consider PTU.
💡 Content filters cannot be fully disabled. Even with custom policies, certain safety guardrails are always active. Design your application accordingly.