Azure-AI_Foundry

Lesson 3: Enterprise Integrations

Azure API Management as AI Gateway

APIM acts as the front door for all AI traffic, providing governance, security, and cost control.

Rate Limiting by Token Count

<inbound>
  <azure-openai-token-limit
    tokens-per-minute="10000"
    counter-key="@(context.Subscription.Id)"
    estimate-prompt-tokens="true"
    remaining-tokens-header-name="x-ratelimit-remaining-tokens" />
</inbound>

Semantic Caching

<inbound>
  <azure-openai-semantic-cache-lookup
    score-threshold="0.9"
    embeddings-backend-id="embeddings-backend" />
</inbound>
<outbound>
  <azure-openai-semantic-cache-store duration="3600" />
</outbound>

Load Balancing

APIM round-robins across configured Azure OpenAI backends with priority/weight. Circuit breaker automatically stops sending traffic to 429-returning backends.

Azure Functions for Serverless AI

import azure.functions as func
from openai import AzureOpenAI
import json

app = func.FunctionApp()
client = AzureOpenAI(
    azure_endpoint="https://my-openai.openai.azure.com/",
    api_version="2024-08-01-preview",
)

@app.route(route="chat", methods=["POST"])
async def chat_function(req: func.HttpRequest) -> func.HttpResponse:
    body = req.get_json()
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": body.get("message", "")},
        ],
        max_tokens=500,
    )
    return func.HttpResponse(
        json.dumps({"reply": response.choices[0].message.content}),
        mimetype="application/json",
    )

Logic Apps & Power Automate

Trigger	AI Action	Output
New email arrives	Summarize with GPT-4o	Post summary to Teams
Form submitted	Classify intent	Route to correct team
File uploaded to SharePoint	Extract data with Doc Intelligence	Write to Dataverse
Scheduled (daily)	Generate report	Email stakeholders

Tip: Use the built-in Azure OpenAI connector in Logic Apps for direct integration.

Microsoft Teams Bots

from botbuilder.core import TurnContext
from openai import AzureOpenAI

client = AzureOpenAI(azure_endpoint="...", api_version="2024-08-01-preview")

async def on_message_activity(turn_context: TurnContext):
    user_text = turn_context.activity.text
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": user_text}],
    )
    await turn_context.send_activity(response.choices[0].message.content)

Microsoft Fabric Integration

Lakehouse - Store and manage training/evaluation data
Data pipelines - ETL for RAG knowledge bases
Notebooks - AI experiments on Fabric Spark clusters
Power BI - Visualize AI metrics and evaluation results

Copilot Studio

Build custom copilots with a low-code experience connecting to AI Foundry models. Deploy to Teams, web, and other channels with built-in analytics.

Integration Decision Guide

Scenario	Recommended Integration
API governance & rate limiting	APIM AI Gateway
Event-driven AI processing	Azure Functions
No-code workflow automation	Logic Apps / Power Automate
Internal employee assistant	Teams Bot + Copilot Studio
Data analytics with AI	Microsoft Fabric
Customer-facing chatbot	Copilot Studio or custom web app