azure-ai-agent-sdk-learning

Azure AI Agent SDK — Core Concepts

Overview

The Azure AI Agent SDK enables you to build AI-powered agents that can reason, use tools, and maintain conversational context. This document covers the foundational concepts you need to understand.

---

Core Concepts

Azure AI Foundry

The unified platform for building, deploying, and managing AI applications on Azure.

• Portal: https://ai.azure.com
• Provides a centralized hub for models, agents, data, and evaluation
• Formerly known as Azure AI Studio

Projects

Workspaces within Azure AI Foundry that contain all your AI resources.

• Each project has a unique connection string used by the SDK
• Projects contain agents, model deployments, data connections, and tools
• Projects map to an underlying Azure resource group
• Format: <endpoint>;<subscription>;<resource-group>;<project-name>

Models

Large Language Models (LLMs) deployed in your Foundry project.

Model	Best For
GPT-4o	Complex reasoning, multi-modal tasks
GPT-4o-mini	Cost-effective general tasks
GPT-4.1	Latest capabilities, improved instruction following
GPT-4.1-mini	Balanced cost and performance

Models are referenced by their deployment name, not the model name itself.

Agents

AI entities that combine a model with instructions and tools to accomplish tasks.

• Have persistent instructions (system prompt) that define behavior
• Are assigned a model deployment for reasoning
• Can use one or more tools to take actions
• Have a unique agent.id used to reference them
• Can be reused across multiple threads/conversations

Threads

Conversation sessions that persist messages and context.

• Represent an ongoing conversation between a user and an agent
• Store the full message history
• Can be reused across multiple runs
• Have a unique thread.id
• Messages are automatically managed — no need to track context windows

Messages

Individual messages within a thread.

• User messages — Input from the user or application
• Assistant messages — Responses generated by the agent
• Can contain text, file references, and annotations
• Messages are ordered chronologically within a thread

Runs

The execution of an agent against a thread.

• When you create a run, the agent reads the thread and generates a response
• Runs go through several statuses: queued → in_progress → completed
• May include intermediate tool calls before completing
• Must be polled or streamed to check for completion
• A run can also end in failed, cancelled, or requires_action status

Tools

Capabilities that agents can use during a run to perform actions beyond text generation.

Code Interpreter

Execute Python code in a sandboxed environment.

• Data analysis and visualization
• Mathematical computations
• File processing (CSV, Excel, PDF)
• Generate charts and images
• Files produced are available as downloadable outputs

File Search

Search uploaded documents using vector embeddings.

• Upload files to a vector store
• Agent automatically chunks, embeds, and indexes documents
• Retrieves relevant passages to answer questions
• Supports PDF, DOCX, TXT, and other formats
• Uses RAG (Retrieval Augmented Generation) under the hood

Function Calling

Invoke custom-defined functions that you implement.

• Define function schemas (name, description, parameters)
• Agent decides when to call functions based on context
• Your code executes the function and returns results
• Enables integration with any external system or API

Azure AI Search

RAG over enterprise knowledge bases using Azure AI Search.

• Connect to existing Azure AI Search indexes
• Enterprise-grade search with semantic ranking
• Supports hybrid search (keyword + vector)
• Ideal for grounding agents in organizational data

OpenAPI

Call external REST APIs via OpenAPI specifications.

• Provide an OpenAPI spec and the agent can call the API
• Agent determines which endpoints to call and with what parameters
• Supports authentication headers and query parameters

Bing Search

Web search grounding for up-to-date information.

• Agent can search the web for current information
• Results are used to ground responses in real-world data
• Requires a Bing Search resource connection

---

Agent Lifecycle

The typical lifecycle of an agent interaction:

Create Agent → Create Thread → Add Message → Create Run → [Tool Calls] → Response → Cleanup

Step-by-Step

1. Create Agent — Define instructions, model, and tools
2. Create Thread — Start a new conversation session
3. Add Message — Add the user’s input to the thread
4. Create Run — Execute the agent against the thread
5. Tool Calls — Agent may invoke tools (code interpreter, functions, etc.)
6. Response — Agent generates its final response as a message
7. Cleanup — Delete agent and thread when no longer needed

Run Status Flow

queued → in_progress → completed
                    ↘ requires_action → (submit tool outputs) → in_progress
                    ↘ failed
                    ↘ cancelled

---

Authentication & Identity

DefaultAzureCredential (Recommended)

The SDK uses DefaultAzureCredential which automatically tries multiple authentication methods in order:

1. Environment variables
2. Managed Identity
3. Azure CLI (az login)
4. Azure PowerShell
5. Interactive browser

from azure.identity import DefaultAzureCredential
credential = DefaultAzureCredential()

var credential = new DefaultAzureCredential();

Azure CLI Auth (Development)

For local development, authenticate via the Azure CLI:

az login
az account set --subscription <subscription-id>

Managed Identity (Production)

For deployed applications, use Managed Identity — no secrets to manage:

• System-assigned — Tied to a specific Azure resource
• User-assigned — Reusable across multiple resources

Required RBAC

The identity running the SDK must have the Azure AI User role (or higher) on the AI Foundry project resource.

az role assignment create \
  --assignee <principal-id> \
  --role "Azure AI User" \
  --scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.MachineLearningServices/workspaces/<project>

---

Key Design Patterns

Single Agent with Multiple Tools

One agent configured with several tools to handle diverse tasks. The agent decides which tool to use based on the user’s request.

Use when: You need a general-purpose assistant that can handle varied requests.

Multi-Agent Orchestration

Multiple specialized agents, each with focused instructions and tools, coordinated by your application logic.

Use when: Different tasks require different expertise, models, or tool configurations.

RAG (Retrieval Augmented Generation)

Agents grounded in your data using File Search or Azure AI Search.

Use when: Agents need to answer questions based on your documents or knowledge bases.

Function Calling for External Integrations

Agents that call your custom functions to interact with databases, APIs, or business logic.

Use when: Agents need to take real-world actions (create tickets, query databases, send emails).

Streaming for Real-Time UX

Stream agent responses token-by-token for responsive user interfaces.

Use when: Building interactive chat UIs where perceived latency matters.

---

Next Steps

• Architecture Reference — Understand the system architecture
• Troubleshooting Guide — Solve common issues