Azure-AI_Foundry

Lesson 2: Fine-Tuning Workflow

Official docs: Create a fine-tuning job

This lesson covers the end-to-end workflow: upload data, create a job, monitor training, deploy, and run inference.

---

1. Full Python Workflow

"""End-to-end fine-tuning workflow with the Azure OpenAI Python SDK."""
import os
import time
from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2025-03-01-preview",
)

# -- Step 1: Upload training & validation files --

training_file = client.files.create(
    file=open("train.jsonl", "rb"),
    purpose="fine-tune",
)
print(f"Training file uploaded: {training_file.id} (status: {training_file.status})")

validation_file = client.files.create(
    file=open("val.jsonl", "rb"),
    purpose="fine-tune",
)
print(f"Validation file uploaded: {validation_file.id}")

# Wait for file processing
for file_obj in [training_file, validation_file]:
    while True:
        status = client.files.retrieve(file_obj.id).status
        if status == "processed":
            break
        elif status == "error":
            raise RuntimeError(f"File {file_obj.id} processing failed.")
        print(f"  File {file_obj.id}: {status}...")
        time.sleep(5)

# -- Step 2: Create fine-tuning job --

job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    validation_file=validation_file.id,
    model="gpt-4o-mini-2024-07-18",
    hyperparameters={
        "n_epochs": 3,
        "batch_size": "auto",
        "learning_rate_multiplier": "auto",
    },
    suffix="contoso-support",          # custom name suffix for the model
)
print(f"Fine-tuning job created: {job.id}")

# -- Step 3: Monitor progress --

while True:
    job = client.fine_tuning.jobs.retrieve(job.id)
    print(f"  Status: {job.status}")

    if job.status == "running":
        # Fetch recent training events
        events = client.fine_tuning.jobs.list_events(job.id, limit=3)
        for event in events.data:
            print(f"    [{event.created_at}] {event.message}")

    if job.status in ("succeeded", "failed", "cancelled"):
        break
    time.sleep(60)

if job.status != "succeeded":
    raise RuntimeError(f"Job ended with status: {job.status} -- {job.error}")

fine_tuned_model = job.fine_tuned_model
print(f"\nFine-tuned model: {fine_tuned_model}")

# -- Step 4: Deploy the model (via REST -- SDK deployment varies) --

# Option A: Deploy via Azure CLI (run in terminal):
#   az cognitiveservices account deployment create \
#     --resource-group <rg> \
#     --name <resource> \
#     --deployment-name contoso-support-ft \
#     --model-name <fine_tuned_model> \
#     --model-version "1" \
#     --model-format OpenAI \
#     --sku-capacity 1 \
#     --sku-name "Standard"

# Option B: Deploy via Python + requests (REST API):
import requests

subscription_id = os.environ["AZURE_SUBSCRIPTION_ID"]
resource_group = os.environ["AZURE_RESOURCE_GROUP"]
resource_name = os.environ["AZURE_OPENAI_RESOURCE_NAME"]
deployment_name = "contoso-support-ft"
mgmt_token = os.environ["AZURE_MGMT_TOKEN"]  # from az account get-access-token

deploy_url = (
    f"https://management.azure.com/subscriptions/{subscription_id}"
    f"/resourceGroups/{resource_group}"
    f"/providers/Microsoft.CognitiveServices/accounts/{resource_name}"
    f"/deployments/{deployment_name}"
    f"?api-version=2024-10-01"
)

deploy_body = {
    "sku": {"name": "Standard", "capacity": 1},
    "properties": {
        "model": {
            "format": "OpenAI",
            "name": fine_tuned_model,
            "version": "1",
        }
    },
}

resp = requests.put(
    deploy_url,
    headers={
        "Authorization": f"Bearer {mgmt_token}",
        "Content-Type": "application/json",
    },
    json=deploy_body,
)
print(f"Deploy response: {resp.status_code}")

# -- Step 5: Use the fine-tuned model --

response = client.chat.completions.create(
    model=deployment_name,   # use deployment name, not the model ID
    messages=[
        {"role": "system", "content": "You are a customer service agent for Contoso Electronics."},
        {"role": "user", "content": "I want to return my headphones."},
    ],
    temperature=0.3,
    max_tokens=300,
)

print(f"\nAssistant: {response.choices[0].message.content}")

---

2. Hyperparameters

Parameter	Default	Range	Guidance
n_epochs	auto (typically 3)	1-50	Start with auto. Increase for small datasets; decrease if validation loss rises.
batch_size	auto	1-256	Larger = faster but uses more memory. auto picks based on dataset size.
learning_rate_multiplier	auto	0.01-5.0	Lower = more conservative. Increase if model is not learning enough; decrease if overfitting.

How Azure Picks auto Defaults

n_epochs:
  - If dataset <= 100 examples  -> ~10-15 epochs
  - If dataset <= 1000 examples -> ~3-4 epochs
  - If dataset > 1000 examples  -> ~2 epochs

batch_size:
  - Typically 1-4 for small datasets
  - Scales with dataset size

learning_rate_multiplier:
  - Scales inversely with batch_size

Tip: Always start with auto for all three. Only tune manually after reviewing training metrics.

---

3. Overfitting Detection

Overfitting occurs when the model memorizes training data rather than learning generalizable patterns.

Signs of Overfitting

Metric	Healthy	Overfitting
Training loss	Decreasing steadily	Decreasing to near zero
Validation loss	Decreasing or stable	Increasing after initial decrease
Gap (train - val loss)	Small gap	Large and growing gap
Output diversity	Varied responses	Repetitive / verbatim training data

Mitigation Strategies

1. Reduce n_epochs        -> Stop training earlier
2. Increase dataset size  -> More diverse examples
3. Lower LR multiplier    -> Smaller weight updates
4. Add validation file    -> Monitor val loss to detect early
5. Review data quality    -> Remove duplicates, fix inconsistencies

Monitoring with Events

"""Check training metrics from job events."""
events = client.fine_tuning.jobs.list_events(job.id, limit=50)

for event in events.data:
    if "training" in event.message.lower() or "validation" in event.message.lower():
        print(f"[Step] {event.message}")

---

4. Deploying Fine-Tuned Models

Option A: Azure Portal

1. Go to Azure AI Foundry -> your resource -> Deployments
2. Click + Create deployment
3. Select your fine-tuned model from the model list (prefixed with ft:)
4. Set deployment name, SKU capacity (TPM)
5. Click Create

Option B: Azure CLI

# Deploy the fine-tuned model
az cognitiveservices account deployment create \
  --resource-group myResourceGroup \
  --name myOpenAIResource \
  --deployment-name contoso-support-ft \
  --model-name "ft:gpt-4o-mini-2024-07-18:contoso-support:abc123" \
  --model-version "1" \
  --model-format OpenAI \
  --sku-capacity 1 \
  --sku-name "Standard"

# Verify the deployment
az cognitiveservices account deployment show \
  --resource-group myResourceGroup \
  --name myOpenAIResource \
  --deployment-name contoso-support-ft

Option C: REST API

# Deploy via ARM REST API
curl -X PUT \
  "https://management.azure.com/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.CognitiveServices/accounts/{resource}/deployments/{deploymentName}?api-version=2024-10-01" \
  -H "Authorization: Bearer $MGMT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "sku": {"name": "Standard", "capacity": 1},
    "properties": {
      "model": {
        "format": "OpenAI",
        "name": "ft:gpt-4o-mini-2024-07-18:contoso-support:abc123",
        "version": "1"
      }
    }
  }'

Tip: Fine-tuned model weights persist even after deleting a deployment. You can redeploy later without retraining.

---

5. Job Management

List All Fine-Tuning Jobs

jobs = client.fine_tuning.jobs.list()
for j in jobs.data:
    print(f"{j.id}  model={j.model}  status={j.status}  "
          f"fine_tuned_model={j.fine_tuned_model}")

Cancel a Running Job

client.fine_tuning.jobs.cancel("ftjob-abc123")
print("Job cancelled.")

Delete a Fine-Tuned Model

# Delete the fine-tuned model (removes weights -- cannot be undone)
client.models.delete("ft:gpt-4o-mini-2024-07-18:contoso-support:abc123")
print("Model deleted.")

List Uploaded Files

files = client.files.list(purpose="fine-tune")
for f in files.data:
    print(f"{f.id}  {f.filename}  {f.status}  {f.bytes} bytes")

Delete an Uploaded File

client.files.delete("file-abc123")

---

6. Complete Workflow Summary

+--------------+     +--------------+     +--------------+
|  Prepare     |---->|  Upload      |---->|  Create Job  |
|  JSONL data  |     |  files       |     |              |
+--------------+     +--------------+     +------+-------+
                                                 |
                                                 v
+--------------+     +--------------+     +--------------+
|  Inference   |<----|  Deploy      |<----|  Monitor     |
|              |     |  model       |     |  training    |
+--------------+     +--------------+     +--------------+

Step	Method	Key SDK Call
Upload	SDK	client.files.create(file, purpose="fine-tune")
Create Job	SDK	client.fine_tuning.jobs.create(...)
Monitor	SDK	client.fine_tuning.jobs.retrieve(job_id)
List Events	SDK	client.fine_tuning.jobs.list_events(job_id)
Deploy	CLI / REST	az cognitiveservices account deployment create
Inference	SDK	client.chat.completions.create(model=deployment_name)
Cleanup	SDK	client.models.delete(model_id)

---

Next –> Lesson 3: Advanced Fine-Tuning