azure-ai-agent-sdk-learning

Azure AI Agent SDK — Architecture Reference

High-Level Architecture

┌─────────────────────────────────────────────────────┐
│                   Your Application                   │
│              (Python / .NET SDK Client)              │
└──────────────────────┬──────────────────────────────┘
                       │ HTTPS/REST
┌──────────────────────▼──────────────────────────────┐
│              Azure AI Foundry Project                │
│  ┌──────────┐ ┌──────────┐ ┌───────────────────┐   │
│  │  Agent 1  │ │  Agent 2  │ │    Connections    │   │
│  │ (GPT-4o) │ │ (GPT-4o) │ │  - AI Search      │   │
│  │ + Tools  │ │ + Tools  │ │  - Storage        │   │
│  └────┬─────┘ └────┬─────┘ │  - Custom APIs    │   │
│       │             │       └───────────────────┘   │
│  ┌────▼─────────────▼────────────────────────────┐  │
│  │              Agent Runtime                     │  │
│  │  ┌─────────┐ ┌───────────┐ ┌──────────────┐  │  │
│  │  │ Threads │ │   Runs    │ │ Tool Executor│  │  │
│  │  │ & Msgs  │ │ & Polling │ │              │  │  │
│  │  └─────────┘ └───────────┘ └──────────────┘  │  │
│  └───────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘
                       │
        ┌──────────────┼──────────────┐
        ▼              ▼              ▼
┌──────────────┐┌──────────────┐┌──────────────┐
│ Code         ││ Azure AI     ││ External     │
│ Interpreter  ││ Search       ││ APIs         │
│              ││ (RAG)        ││ (OpenAPI)    │
└──────────────┘└──────────────┘└──────────────┘

---

SDK Architecture

Client Hierarchy

The SDK follows a layered client pattern:

AIProjectClient                          (top-level client)
  ├── .agents                            (agents namespace / sub-client)
  │     ├── create_agent()               → Agent
  │     ├── create_thread()              → Thread
  │     ├── create_message()             → Message
  │     ├── create_run()                 → Run
  │     ├── create_and_process_run()     → Run (with automatic polling)
  │     └── create_stream()             → Stream
  └── .connections / .telemetry          (other sub-clients)

Python

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

# Top-level client
project_client = AIProjectClient.from_connection_string(
    credential=DefaultAzureCredential(),
    conn_str="<connection-string>"
)

# Access the agents sub-client
agents_client = project_client.agents

.NET

var projectClient = new AIProjectClient(connectionString, new DefaultAzureCredential());

// Access the agents sub-client
AgentsClient agentsClient = projectClient.GetAgentsClient();

Thread Management

Threads are lightweight conversation containers:

Thread
  ├── Message 1 (role: user)      "Analyze this CSV file"
  ├── Message 2 (role: assistant) "I'll analyze the data..."
  ├── Message 3 (role: user)      "Show me a chart"
  └── Message 4 (role: assistant) "Here's the visualization..." + [image file]

• Threads are persistent — they survive across runs
• Messages accumulate in chronological order
• The agent sees the full thread history on each run
• You can add multiple user messages before starting a run

Run Lifecycle and Polling

A run progresses through states:

                    ┌──────────────────────────────────┐
                    │                                  │
  create_run() → queued → in_progress ──→ completed   │
                              │                        │
                              ├──→ requires_action ────┘
                              │    (function calls)   submit_tool_outputs()
                              │
                              ├──→ failed
                              └──→ cancelled

Polling approach (create_and_process_run):

• SDK handles polling internally
• Blocks until the run reaches a terminal state
• Simplest approach for most use cases

Manual polling:

• Call create_run() then poll get_run() in a loop
• Provides more control over timeout and cancellation

Streaming approach (create_stream):

• Events are pushed to your code as they occur
• Best for real-time UIs
• Handles tool calls via event callbacks

Tool Execution Flow

When an agent decides to use a tool:

1. Agent receives thread context
2. Agent decides a tool call is needed
3. Run enters "requires_action" state (for function calling)
   — or tool executes server-side (code interpreter, file search)
4. For function calling:
   a. Your code receives the function name + arguments
   b. Your code executes the function
   c. Your code submits the result via submit_tool_outputs()
   d. Run resumes processing
5. Agent incorporates tool results into its response
6. Run reaches "completed" state

---

Production Architecture

Recommended Production Setup

┌─────────────────────────────────────────────────────────────┐
│                        Azure Cloud                          │
│                                                             │
│  ┌───────────────┐    ┌──────────────────────────────────┐  │
│  │ Azure Front   │    │        VNet                       │  │
│  │ Door / APIM   │───▶│  ┌────────────────────────────┐  │  │
│  └───────────────┘    │  │   App Service / Container  │  │  │
│                       │  │   Apps (your application)   │  │  │
│                       │  │   + Managed Identity        │  │  │
│                       │  └──────────┬─────────────────┘  │  │
│                       │             │                     │  │
│                       │  ┌──────────▼─────────────────┐  │  │
│                       │  │  AI Foundry Project         │  │  │
│                       │  │  (Private Endpoint)         │  │  │
│                       │  └────────────────────────────┘  │  │
│                       └──────────────────────────────────┘  │
│                                                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │ App Insights │  │  Key Vault   │  │ Azure AI Search  │  │
│  │ (monitoring) │  │  (secrets)   │  │ (knowledge base) │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Authentication (Managed Identity)

In production, use Managed Identity — no secrets to manage or rotate.

# In production, DefaultAzureCredential picks up Managed Identity automatically
credential = DefaultAzureCredential()
project_client = AIProjectClient.from_connection_string(
    credential=credential,
    conn_str=os.environ["AZURE_AI_CONNECTION_STRING"]
)

Assign the required RBAC role to the managed identity:

az role assignment create \
  --assignee <managed-identity-principal-id> \
  --role "Azure AI User" \
  --scope <ai-foundry-project-resource-id>

Monitoring with Application Insights

Integrate Application Insights for observability:

• Traces — Track agent interactions, tool calls, and latencies
• Metrics — Token usage, run duration, error rates
• Logs — Detailed request/response logging for debugging

from azure.monitor.opentelemetry import configure_azure_monitor

configure_azure_monitor(connection_string=os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"])

Secrets Management with Key Vault

Store sensitive configuration in Azure Key Vault:

• Connection strings
• API keys for external services
• OpenAPI authentication credentials

API Gateway (Azure API Management / Front Door)

Place an API gateway in front of your application for:

• Rate limiting and throttling
• Authentication and authorization
• Request/response transformation
• Caching and load balancing

CI/CD Pipeline

Automate deployment with Azure DevOps or GitHub Actions:

Code Push → Build → Test → Deploy Infrastructure (Bicep/Terraform) → Deploy Application

Key stages:

1. Build — Package application, run unit tests
2. Infrastructure — Deploy/update Azure resources via IaC
3. Deploy — Push application to App Service / Container Apps
4. Validate — Run integration tests against deployed environment

---

Security Considerations

RBAC and Least Privilege

• Grant only the minimum required roles
• Use Azure AI User for applications that need agent access
• Use Azure AI Developer for users who create and manage agents
• Avoid Owner or Contributor unless necessary for infrastructure management

Network Security

• Deploy AI Foundry projects with Private Endpoints
• Use VNet integration for App Service / Container Apps
• Restrict inbound access with Network Security Groups (NSGs)
• Use Azure Front Door or API Management as the public entry point

Data Encryption

• At rest — All data is encrypted by default using Microsoft-managed keys (or customer-managed keys via Key Vault)
• In transit — All SDK communication uses TLS 1.2+
• Thread data — Messages and files are stored securely within the Foundry project

Responsible AI Guardrails

• Configure content filters on model deployments to block harmful content
• Use agent instructions to define boundaries and acceptable behavior
• Implement application-level validation on agent outputs
• Monitor agent interactions for policy compliance
• Use Azure AI Content Safety for additional content moderation

---

Next Steps

• Core Concepts — Understand the building blocks
• Troubleshooting Guide — Solve common issues