Azure-AI_Foundry

Lesson 3: Multi-Agent Orchestration

Learning Objectives

• Understand multi-agent patterns: handoff, delegation, and collaboration
• Use Semantic Kernel for agent orchestration in Python
• Explore the AutoGen framework for complex multi-agent workflows
• Apply safety guardrails and human-in-the-loop controls

---

Why Multi-Agent Systems?

A single agent works well for focused tasks, but real-world applications often require:

• Specialization: A coding agent, a research agent, and a reviewer agent each excel at different tasks
• Separation of concerns: Each agent has a clear role with scoped instructions
• Parallel processing: Multiple agents can work on sub-tasks simultaneously
• Quality control: One agent checks another’s output

---

Multi-Agent Patterns

1. Handoff Pattern

One agent transfers the conversation to another when it detects a topic outside its expertise.

User -> Triage Agent -> (routes to) -> Sales Agent OR Support Agent

2. Delegation Pattern

A manager agent breaks a complex task into sub-tasks and delegates them to specialist agents.

User -> Manager Agent -> Research Agent (gathers data)
                      -> Writer Agent (drafts content)
                      -> Reviewer Agent (checks quality)

3. Collaboration Pattern

Multiple agents interact in a shared conversation thread, each contributing their expertise.

User -> [Debate Thread] <- Agent A (pro arguments)
                        <- Agent B (con arguments)
                        <- Moderator Agent (synthesizes)

---

Semantic Kernel Agent Orchestration

Semantic Kernel provides built-in abstractions for building multi-agent systems on Azure:

from semantic_kernel.agents import ChatCompletionAgent, AgentGroupChat
from semantic_kernel.agents.strategies import (
    TerminationStrategy, SequentialSelectionStrategy,
)
from semantic_kernel import Kernel

kernel = Kernel()

researcher = ChatCompletionAgent(
    kernel=kernel,
    name="Researcher",
    instructions=(
        "You are a research specialist. Gather facts and data "
        "relevant to the topic. Cite sources when possible."
    ),
)

writer = ChatCompletionAgent(
    kernel=kernel,
    name="Writer",
    instructions=(
        "You are a technical writer. Take the researcher's findings "
        "and produce a clear, well-structured summary."
    ),
)

reviewer = ChatCompletionAgent(
    kernel=kernel,
    name="Reviewer",
    instructions=(
        "You review the writer's output for accuracy and clarity. "
        "Respond with 'APPROVED' if the content is ready, or provide "
        "specific feedback for revision."
    ),
)

class ApprovalTermination(TerminationStrategy):
    async def should_agent_terminate(self, agent, history):
        last = history[-1].content if history else ""
        return "APPROVED" in last and agent.name == "Reviewer"

group_chat = AgentGroupChat(
    agents=[researcher, writer, reviewer],
    selection_strategy=SequentialSelectionStrategy(),
    termination_strategy=ApprovalTermination(
        agents=[reviewer], maximum_iterations=6
    ),
)

await group_chat.add_chat_message(
    message="Explain the benefits of edge computing for IoT."
)

async for message in group_chat.invoke():
    print(f"[{message.name}]: {message.content[:200]}")

---

AutoGen Multi-Agent Framework

AutoGen provides a framework for building complex multi-agent workflows with support for human-in-the-loop, tool use, and code execution:

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination
from autogen_ext.models.openai import AzureOpenAIChatCompletionClient

model_client = AzureOpenAIChatCompletionClient(
    azure_deployment="gpt-4o",
    azure_endpoint="https://your-resource.openai.azure.com/",
    model="gpt-4o",
    api_version="2024-06-01",
)

planner = AssistantAgent(
    name="Planner",
    model_client=model_client,
    system_message="Break complex tasks into steps. Assign to Coder or Analyst.",
)

coder = AssistantAgent(
    name="Coder",
    model_client=model_client,
    system_message="Write Python code to solve assigned tasks.",
)

termination = TextMentionTermination("TASK_COMPLETE")
team = RoundRobinGroupChat([planner, coder], termination_condition=termination)

result = await team.run(task="Create a Python function to calculate compound interest.")

---

Agent Safety: Guardrails and Controls

Input Guardrails

Validate user input before it reaches the agent:

• Content filtering for harmful prompts
• Schema validation for structured inputs
• Rate limiting to prevent abuse

Output Guardrails

Check agent responses before returning to users:

• PII detection and redaction
• Factuality checks against known data
• Tone and brand compliance validation

Human-in-the-Loop

For high-stakes decisions, require human approval:

if run.status == "requires_action":
    pending_actions = run.required_action.submit_tool_outputs.tool_calls
    for action in pending_actions:
        if action.function.name in ("approve_refund", "delete_account"):
            human_decision = await request_human_approval(action)
            if not human_decision.approved:
                client.agents.cancel_run(thread_id=thread.id, run_id=run.id)
                break

Tip: Start with a single agent and add multi-agent orchestration only when you hit clear limitations. Complexity grows fast with multiple agents.

---

Best Practices for Multi-Agent Systems

1. Clear role boundaries: Each agent should have a distinct, non-overlapping responsibility
2. Limit iterations: Set maximum iteration counts to prevent infinite loops
3. Log everything: Capture all inter-agent messages for debugging and auditing
4. Fail gracefully: If one agent fails, the orchestrator should handle it without crashing
5. Test in isolation first: Verify each agent works independently before combining them
6. Use structured outputs: Agents communicate more reliably with JSON schemas

---

Key Takeaways

• Multi-agent patterns (handoff, delegation, collaboration) solve different orchestration needs
• Semantic Kernel and AutoGen provide production-ready frameworks for agent orchestration
• Always implement safety guardrails: input validation, output filtering, and human-in-the-loop
• Start simple – add agents only when a single agent cannot handle the task complexity