Azure-AI_Foundry

Module 7: Fine-Tuning Azure OpenAI Models

Official docs: Fine-tuning Azure OpenAI models

Why Fine-Tune?

Fine-tuning customizes a foundation model with your own data, producing a model that follows your style, terminology, and reasoning patterns — without stuffing everything into the prompt.

---

Decision Framework

Before investing in fine-tuning, walk through this decision ladder:

+--------------------------------------------------+
|            Can prompt engineering solve it?       |
|  (system message, few-shot examples, CoT)        |
+-------------------------+------------------------+
                          | No
                          v
+--------------------------------------------------+
|         Can RAG solve it?                        |
|  (retrieve context from your knowledge base)     |
+-------------------------+------------------------+
                          | No
                          v
+--------------------------------------------------+
|         Fine-tune the model                      |
|  (teach the model new behavior / style / format) |
+--------------------------------------------------+

When to Fine-Tune vs. When Not To

When to Fine-Tune	When NOT to Fine-Tune
Consistent output format/style across thousands of calls	You just need access to external knowledge (use RAG)
Reduce token usage by eliminating long system prompts	Your data changes frequently (RAG is more flexible)
Teach domain-specific terminology or jargon	You have fewer than 50 high-quality examples
Improve reliability on a narrow, well-defined task	A well-crafted system prompt already achieves 90%+ accuracy
Reproduce a specific tone of voice at scale	Need the model to cite sources (RAG + grounding)
Lower latency by removing few-shot examples from the prompt	Want to add new factual knowledge (fine-tuning can hallucinate)
Distill a larger model’s behavior into a smaller one	Budget or timeline doesn’t allow iterative training runs

---

Supported Models

Model	Fine-Tuning Support	Min Examples	Recommended Examples	Context Window
gpt-4o (2024-08-06)	GA	10	50-100+	128k
gpt-4o-mini (2024-07-18)	GA	10	50-100+	128k
gpt-4.1 (2025-04-14)	GA	10	50-100+	1M
gpt-4.1-mini (2025-04-14)	GA	10	50-100+	1M
gpt-4.1-nano (2025-04-14)	GA	10	50-100+	1M
gpt-3.5-turbo (0125)	GA	10	50-100+	16k

Tip: Check Azure OpenAI models for the latest regional availability. Not all models are available in every region.

---

Cost Considerations

Fine-tuning costs have three components:

Component	Description	Billed Per
Training	Compute for the training run	1K tokens processed x epochs
Hosting	Keeping your fine-tuned model deployed	Hour (while deployed)
Inference	Using the model for completions	1K tokens (input + output)

Cost Formula

Training cost = (tokens_in_dataset x n_epochs x price_per_1K_training_tokens) / 1000
Hosting cost  = hours_deployed x price_per_hour
Inference cost = (input_tokens + output_tokens) x price_per_1K_inference_tokens / 1000

Example Estimate

Dataset: 100 examples x ~500 tokens each = 50,000 tokens
Epochs: 3
Total training tokens: 50,000 x 3 = 150,000 tokens

If training price = $0.008 / 1K tokens (gpt-4o-mini):
  Training cost = 150 x $0.008 = $1.20

Hosting: $1.70/hr x 24 hrs = $40.80/day (varies by model)

Tip: Delete deployments when not in use to avoid hosting charges. Fine-tuned model weights are retained even after un-deploying.

---

Quick Start: Fine-Tuning with Python

"""Minimal fine-tuning workflow using the Azure OpenAI Python SDK."""
import time
from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://<your-resource>.openai.azure.com/",
    api_key="<your-api-key>",          # or use DefaultAzureCredential
    api_version="2025-03-01-preview",
)

# 1. Upload training file
training_file = client.files.create(
    file=open("training_data.jsonl", "rb"),
    purpose="fine-tune",
)
print(f"Uploaded file: {training_file.id}")

# 2. Create fine-tuning job
job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    model="gpt-4o-mini-2024-07-18",   # base model
    hyperparameters={
        "n_epochs": 3,
        "batch_size": "auto",
        "learning_rate_multiplier": "auto",
    },
)
print(f"Job created: {job.id}")

# 3. Poll until complete
while True:
    job = client.fine_tuning.jobs.retrieve(job.id)
    print(f"Status: {job.status}")
    if job.status in ("succeeded", "failed", "cancelled"):
        break
    time.sleep(30)

# 4. Use the fine-tuned model
if job.status == "succeeded":
    fine_tuned_model = job.fine_tuned_model
    print(f"Fine-tuned model: {fine_tuned_model}")

    response = client.chat.completions.create(
        model=fine_tuned_model,        # e.g. "ft:gpt-4o-mini-2024-07-18:..."
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Summarize this ticket..."},
        ],
    )
    print(response.choices[0].message.content)

---

Module Roadmap

Lesson	Topic	What You’ll Learn
Lesson 1	Data Preparation	JSONL format, quality requirements, validation, augmentation
Lesson 2	Fine-Tuning Workflow	Upload, train, monitor, deploy, use (SDK, CLI, REST)
Lesson 3	Advanced Fine-Tuning	Continuous FT, vision, distillation, W&B integration
Knowledge Check	Quiz	15 scenario-based questions to test your understanding

---

Prerequisites

pip install openai tiktoken

Set your environment variables:

# Linux / macOS
export AZURE_OPENAI_ENDPOINT="https://<resource>.openai.azure.com/"
export AZURE_OPENAI_API_KEY="<key>"

# Windows PowerShell
$env:AZURE_OPENAI_ENDPOINT = "https://<resource>.openai.azure.com/"
$env:AZURE_OPENAI_API_KEY = "<key>"

Next –> Lesson 1: Data Preparation