Azure-AI_Foundry

Azure AI Foundry — Quick Reference Cards

Concise reference tables for day-to-day work with Azure AI Foundry. Last updated: 2025 · Always verify against official docs.

1. Service Comparison: Azure AI Foundry vs Standalone Services

Capability	Azure AI Foundry (ai.azure.com)	Standalone Azure AI Services
Unified portal	✅ Single portal for all AI	❌ Separate portals per service
Model Catalog	✅ 1,700+ models (OpenAI, Meta, Mistral, Microsoft)	❌ Only service-specific models
Hub/Project governance	✅ Centralized management	❌ Per-resource management
Playground	✅ Chat, Completions, Images	⚠️ Limited (Azure OpenAI Studio)
RAG integration	✅ Built-in “Add your data”	❌ Manual integration required
Evaluation tools	✅ Built-in evaluators + SDK	❌ Build your own
Agent Service	✅ Native agent framework	❌ Manual orchestration
Content Safety	✅ Integrated content filters	✅ Standalone Content Safety API
Fine-tuning	✅ In-portal fine-tuning UI	✅ Via Azure OpenAI Service
CI/CD & MLOps	✅ SDK + GitHub/ADO integration	⚠️ Manual pipeline setup
Billing	Per Hub (shared resources)	Per individual resource

When to use Foundry: New AI projects, multi-model scenarios, teams needing governance. When to use standalone: Single-service needs, existing integrations, legacy applications.

📖 What is Azure AI Foundry?

2. Model Deployment Options Comparison

Feature	Serverless API (MaaS)	Managed Compute	Provisioned Throughput (PTU)
Billing	Per token (pay-as-you-go)	Per hour (compute)	Per PTU (reserved capacity)
Models	Select OSS + Azure OpenAI	All supported models	Azure OpenAI models
Scale	Auto-scales	Manual/auto scale	Reserved, consistent
Latency	Variable	Predictable	Lowest, guaranteed
Min Cost	$0 (pay per use)	~$0.37/hr+	~$2/hr+ per PTU
Best For	Dev/test, variable load	Custom models, OSS	Production, high volume
SLA	Standard	Standard	Highest
Cold Start	Possible	Warm	None
Data Residency	Region-bound	Region-bound	Region-bound
Setup Complexity	Low (one-click)	Medium	High (capacity planning)

Decision Guide:

Prototyping / low volume → Serverless API (PAYG)
Predictable production workload → Provisioned Throughput (PTU)
Open-source models at scale → Managed Compute
Cost-sensitive variable workload → Serverless API with rate limits

📖 Model deployment options

3. Decision Tree: RAG vs Fine-Tuning vs Prompt Engineering

Start: What do you need to improve?
│
├─ Model doesn't know about MY data
│   ├─ Data changes frequently? ──────────────→ RAG
│   ├─ Data is static & domain-specific? ────→ Fine-tuning (or RAG)
│   └─ Small amount of context needed? ──────→ Prompt Engineering (few-shot)
│
├─ Model output FORMAT/STYLE needs to change
│   ├─ Can be specified in system prompt? ───→ Prompt Engineering
│   └─ Consistent style across 1000s of calls? → Fine-tuning
│
├─ Model needs to be MORE ACCURATE
│   ├─ Accuracy with source attribution? ────→ RAG (with citations)
│   ├─ Domain-specific accuracy? ────────────→ Fine-tuning + RAG
│   └─ General accuracy? ───────────────────→ Better model + Prompt Engineering
│
└─ Model needs to FOLLOW INSTRUCTIONS better
    ├─ Complex multi-step tasks? ────────────→ Prompt Engineering (chain-of-thought)
    ├─ Consistent behavior pattern? ─────────→ Fine-tuning
    └─ Needs external tools/actions? ────────→ Agents + Function Calling

Approach	When to Use	Cost	Effort	Latency Impact
Prompt Engineering	Always start here	Lowest	Low	None
RAG	Dynamic knowledge, citations needed	Medium	Medium	+200–500ms retrieval
Fine-Tuning	Style/format consistency, reduce tokens	High (training)	High	Slightly lower (fewer tokens)
Combination	Production-grade applications	Highest	Highest	Varies

📖 Fine-tuning considerations

4. SDK Quick Reference

azure-ai-projects

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

# Connect to project
client = AIProjectClient(
    credential=DefaultAzureCredential(),
    endpoint="https://<hub>.api.azureml.ms",
    subscription_id="<sub-id>",
    resource_group_name="<rg>",
    project_name="<project>"
)

# Get inference client
chat = client.inference.get_chat_completions_client()

# Agents
agent = client.agents.create_agent(model="gpt-4o", name="my-agent", instructions="...")
thread = client.agents.create_thread()
message = client.agents.create_message(thread_id=thread.id, role="user", content="Hello")
run = client.agents.create_and_process_run(thread_id=thread.id, agent_id=agent.id)
messages = client.agents.list_messages(thread_id=thread.id)

📖 azure-ai-projects SDK

azure-ai-inference

from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

# Direct model call
client = ChatCompletionsClient(
    endpoint="https://<endpoint>.openai.azure.com/",
    credential=AzureKeyCredential("<key>")
)

# Chat completion
response = client.complete(
    model="gpt-4o",
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="What is Azure AI Foundry?")
    ],
    temperature=0.7,
    max_tokens=800
)
print(response.choices[0].message.content)

# Streaming
for chunk in client.complete(messages=[...], stream=True):
    print(chunk.choices[0].delta.content or "", end="")

📖 azure-ai-inference SDK

azure-ai-evaluation

from azure.ai.evaluation import (
    GroundednessEvaluator,
    RelevanceEvaluator,
    CoherenceEvaluator,
    FluencyEvaluator,
    evaluate
)

# Single evaluator
groundedness = GroundednessEvaluator(model_config)
result = groundedness(
    response="The capital of France is Paris.",
    context="France is a country in Europe. Its capital is Paris."
)

# Batch evaluation
eval_result = evaluate(
    data="test_dataset.jsonl",
    evaluators={
        "groundedness": GroundednessEvaluator(model_config),
        "relevance": RelevanceEvaluator(model_config),
        "coherence": CoherenceEvaluator(model_config),
    },
    output_path="eval_results.json"
)

📖 azure-ai-evaluation SDK

5. REST API Endpoint Patterns

Operation	Method	Endpoint Pattern
Chat Completions	POST	`https://<endpoint>.openai.azure.com/openai/deployments/<deployment>/chat/completions?api-version=2024-10-21`
Completions	POST	`https://<endpoint>.openai.azure.com/openai/deployments/<deployment>/completions?api-version=2024-10-21`
Embeddings	POST	`https://<endpoint>.openai.azure.com/openai/deployments/<deployment>/embeddings?api-version=2024-10-21`
Image Generation	POST	`https://<endpoint>.openai.azure.com/openai/deployments/<deployment>/images/generations?api-version=2024-10-21`
Serverless Model	POST	`https://<deployment>.<region>.models.ai.azure.com/v1/chat/completions`
Fine-Tuning (Create)	POST	`https://<endpoint>.openai.azure.com/openai/fine_tuning/jobs?api-version=2024-10-21`
Fine-Tuning (Status)	GET	`https://<endpoint>.openai.azure.com/openai/fine_tuning/jobs/<job-id>?api-version=2024-10-21`
Content Safety	POST	`https://<endpoint>.cognitiveservices.azure.com/contentsafety/text:analyze?api-version=2024-09-01`

Common Headers:

api-key: <your-api-key>
Content-Type: application/json
Authorization: Bearer <entra-token>   # Alternative to api-key

📖 Azure OpenAI REST API reference

6. Pricing Tier Comparison (Approximate)

⚠️ Prices change frequently. Always verify at azure.microsoft.com/pricing.

Model	Input (per 1M tokens)	Output (per 1M tokens)	Notes
GPT-4o	$2.50	$10.00	Best value for most tasks
GPT-4o mini	$0.15	$0.60	Cost-efficient for simple tasks
GPT-4 Turbo	$10.00	$30.00	Legacy, prefer GPT-4o
o1	$15.00	$60.00	Reasoning model
o1-mini	$3.00	$12.00	Lightweight reasoning
text-embedding-3-large	$0.13	—	3,072 dimensions
text-embedding-3-small	$0.02	—	1,536 dimensions
DALL-E 3 (Standard)	$0.04/image	—	1024×1024

Supporting Service	Free Tier	Standard / Basic
Azure AI Search	Free (50 MB, 3 indexes)	Basic: ~$70/mo
Azure AI Content Safety	Free (5K transactions/mo)	Standard: $1/1K transactions
Azure Blob Storage	5 GB free	~$0.02/GB/mo (Hot)

7. Region Availability Matrix

Check real-time availability: Azure AI Foundry region support

Region	GPT-4o	GPT-4o mini	o1	Llama 3	Mistral	Fine-Tuning	PTU
East US	✅	✅	✅	✅	✅	✅	✅
East US 2	✅	✅	⚠️	✅	✅	✅	✅
West US	✅	✅	❌	✅	⚠️	⚠️	⚠️
West US 3	✅	✅	⚠️	✅	✅	✅	✅
North Central US	✅	✅	❌	⚠️	⚠️	✅	⚠️
Sweden Central	✅	✅	✅	✅	✅	✅	✅
West Europe	✅	✅	⚠️	✅	✅	⚠️	✅
UK South	✅	✅	❌	⚠️	⚠️	⚠️	⚠️
Japan East	✅	✅	❌	⚠️	❌	⚠️	⚠️
Australia East	✅	✅	❌	⚠️	❌	⚠️	⚠️

✅ = Available · ⚠️ = Limited/Preview · ❌ = Not available

📖 Azure OpenAI model availability

8. Common Error Codes and Fixes

Error Code	Message	Cause	Fix
`401`	Unauthorized	Invalid API key or token expired	Regenerate key or refresh `DefaultAzureCredential`
`403`	Forbidden	Missing RBAC role or content filter blocked	Assign `Cognitive Services OpenAI User` role; check content filters
`404`	Resource Not Found	Wrong endpoint, deployment name, or API version	Verify endpoint URL and deployment name in portal
`429`	Rate Limit Exceeded	Too many requests or TPM limit hit	Implement exponential backoff; increase TPM quota
`500`	Internal Server Error	Transient Azure error	Retry with backoff; check Azure Status
`503`	Service Unavailable	Model overloaded or region capacity	Retry; consider PTU or different region
`content_filter`	Content filtered	Input/output triggered content safety filter	Adjust content filter settings or modify prompt
`context_length_exceeded`	Context too long	Input + output exceeds model context window	Reduce input tokens; use a model with larger context
`DeploymentNotFound`	Deployment does not exist	Deployment name mismatch or not yet provisioned	Check deployment name in portal; wait for provisioning
`QuotaExceeded`	Insufficient quota	Subscription/region quota exhausted	Request quota increase in portal or use different region

📖 Azure OpenAI error codes

9. RBAC Roles for Azure AI Foundry

Role	Scope	Permissions
Azure AI Developer	Hub/Project	Create deployments, run experiments, use playground, manage data. Cannot manage Hub settings.
Azure AI Inference Deployment Operator	Hub/Project	Deploy models only. Cannot access data or run experiments.
Cognitive Services OpenAI User	Azure OpenAI resource	Call OpenAI APIs (chat, completions, embeddings). Read-only.
Cognitive Services OpenAI Contributor	Azure OpenAI resource	Create deployments, upload fine-tuning data, manage models.
Cognitive Services Contributor	AI Services resource	Full access to AI Services (speech, vision, language, etc.).
Contributor	Resource Group/Hub	Full resource management. Cannot manage RBAC.
Owner	Resource Group/Hub	Full resource management + RBAC management.
Reader	Any scope	Read-only access. Cannot call APIs or create resources.
Storage Blob Data Contributor	Storage Account	Read/write blob data. Required for file uploads and data access.
Key Vault Secrets User	Key Vault	Read secrets. Required for connection strings and API keys.
Search Index Data Reader	AI Search	Query search indexes. Required for RAG scenarios.
Search Index Data Contributor	AI Search	Create and manage search indexes and data.

Minimum roles for a developer to use AI Foundry:

Azure AI Developer on the Hub or Project
Cognitive Services OpenAI User on the AI Services resource
Storage Blob Data Contributor on the Storage Account
Key Vault Secrets User on the Key Vault

📖 Azure AI Foundry RBAC roles

10. Quotas and Limits Reference

Limit	Default Value	Can Increase?
Hubs per subscription	50	Yes (support ticket)
Projects per Hub	100	Yes
Model deployments per project	50	Yes
GPT-4o TPM (Tokens Per Minute)	80K–450K	Yes (quota page)
GPT-4o mini TPM	450K–2M	Yes
Requests per minute (RPM)	Varies by model	Yes
Max context window (GPT-4o)	128K tokens	No
Max output tokens (GPT-4o)	16K tokens	No
Fine-tuning training file size	512 MB	No
Fine-tuning concurrent jobs	3	Yes
Embeddings batch size	2,048	No
Content filter transactions/min	1,000	Yes
AI Search indexes (Basic)	15	Upgrade tier
AI Search document size	16 MB	No

Check your current quotas:

# Via Azure CLI
az cognitiveservices usage list --location eastus

# Via Portal
# ai.azure.com → Settings → Quotas

📖 Azure OpenAI quotas and limits 📖 Azure AI Search limits