Module 10: MLOps & Deployment for Generative AI
Overview
MLOps for generative AI extends traditional ML lifecycle practices to handle the unique challenges of large language models, prompt management, and AI-powered applications built on Azure AI Foundry.
The Generative AI Lifecycle
Develop --> Evaluate --> Deploy --> Monitor
| |
+--------- Feedback Loop ----------+
Develop: Prompt design, fine-tuning, RAG
Evaluate: Auto evals, human review, safety
Deploy: MaaS, MaaP, real-time, batch
Monitor: Logs, drift, cost, performance
Key Differences from Traditional MLOps
| Aspect |
Traditional MLOps |
GenAI MLOps |
| Artifacts |
Model binaries |
Prompts, configs, fine-tuned adapters |
| Evaluation |
Accuracy, F1 |
Groundedness, coherence, safety |
| Versioning |
Model weights |
Prompt templates + model versions |
| Deployment |
Custom containers |
Managed endpoints (MaaS/MaaP) |
| Monitoring |
Data drift |
Content safety, token usage, latency |
Infrastructure as Code for AI Foundry
// main.bicep - AI Foundry hub and project
resource aiHub 'Microsoft.MachineLearningServices/workspaces@2024-04-01' = {
name: 'my-ai-hub'
location: resourceGroup().location
kind: 'Hub'
identity: { type: 'SystemAssigned' }
properties: {
friendlyName: 'Production AI Hub'
storageAccount: storageAccount.id
keyVault: keyVault.id
}
}
resource aiProject 'Microsoft.MachineLearningServices/workspaces@2024-04-01' = {
name: 'my-ai-project'
location: resourceGroup().location
kind: 'Project'
identity: { type: 'SystemAssigned' }
properties: {
friendlyName: 'GenAI App Project'
hubResourceId: aiHub.id
}
}
Module Lessons
| Lesson |
Topic |
Focus |
| 1 |
Endpoint Management |
Real-time, batch, scaling, A/B testing |
| 2 |
CI/CD Pipelines |
GitHub Actions, evaluation gates, IaC |
| 3 |
Versioning & Rollback |
Blue-green, canary, disaster recovery |
Learning Objectives
- Design an end-to-end MLOps pipeline for generative AI applications
- Deploy and manage real-time and batch endpoints
- Implement CI/CD with automated evaluation gates
- Apply versioning, rollback, and disaster recovery strategies
- Use Infrastructure as Code to provision AI Foundry resources
References