Official docs: Fine-tuning Azure OpenAI
This lesson covers advanced techniques: continuous fine-tuning, vision fine-tuning, model distillation, and Weights & Biases integration.
Continuous fine-tuning lets you incrementally improve a previously fine-tuned model with new data – without starting from scratch.
"""Continuous fine-tuning: train on top of a previously fine-tuned model."""
import os
import time
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2025-03-01-preview",
)
# Upload new incremental training data
new_file = client.files.create(
file=open("new_training_data.jsonl", "rb"),
purpose="fine-tune",
)
# Fine-tune starting from your EXISTING fine-tuned model
job = client.fine_tuning.jobs.create(
training_file=new_file.id,
model="ft:gpt-4o-mini-2024-07-18:contoso-support:abc123", # previous FT model
hyperparameters={
"n_epochs": 2, # fewer epochs for incremental updates
"learning_rate_multiplier": 0.5, # conservative LR to avoid catastrophic forgetting
},
suffix="contoso-support-v2",
)
print(f"Continuous FT job: {job.id}")
# Monitor
while True:
job = client.fine_tuning.jobs.retrieve(job.id)
if job.status in ("succeeded", "failed", "cancelled"):
break
print(f" {job.status}...")
time.sleep(60)
if job.status == "succeeded":
print(f"Updated model: {job.fine_tuned_model}")
Warning: Each continuous fine-tuning creates a new model. Keep track of your model lineage (v1 -> v2 -> v3) and test each version before deploying.
| Practice | Why |
|---|---|
| Use a lower learning rate (0.2-0.5x) | Prevents catastrophic forgetting of earlier training |
| Use fewer epochs (1-2) | New data is incremental, not a full retrain |
| Include some original examples | Reinforces prior learning (mix ~20% old data) |
| Version your models with suffixes | Track lineage: v1, v2, v3 |
| A/B test before swapping in production | Validate new model does not regress |
Fine-tune models like gpt-4o to handle image + text inputs for domain-specific visual tasks (e.g., medical imaging, product inspection, document parsing).
{"messages": [{"role": "system", "content": "You are a product quality inspector."}, {"role": "user", "content": [{"type": "text", "text": "Inspect this product image for defects."}, {"type": "image_url", "image_url": {"url": "https://your-storage.blob.core.windows.net/images/product_001.jpg"}}]}, {"role": "assistant", "content": "No defects detected. The product surface is smooth, labeling is correctly positioned, and packaging is intact."}]}
{"messages": [{"role": "system", "content": "You are a product quality inspector."}, {"role": "user", "content": [{"type": "text", "text": "Inspect this product image for defects."}, {"type": "image_url", "image_url": {"url": "https://your-storage.blob.core.windows.net/images/product_002.jpg"}}]}, {"role": "assistant", "content": "Defect found: scratch on the top-left corner of the device casing, approximately 2cm long. Recommend rejecting this unit."}]}
"""Prepare vision fine-tuning data from a folder of labeled images."""
import json
import base64
from pathlib import Path
def create_vision_example_url(
image_url: str,
instruction: str,
expected_response: str,
system_prompt: str = "You are a product quality inspector.",
) -> dict:
"""Create a vision fine-tuning example using an image URL."""
return {
"messages": [
{"role": "system", "content": system_prompt},
{
"role": "user",
"content": [
{"type": "text", "text": instruction},
{"type": "image_url", "image_url": {"url": image_url}},
],
},
{"role": "assistant", "content": expected_response},
]
}
def create_vision_example_base64(
image_path: str,
instruction: str,
expected_response: str,
system_prompt: str = "You are a product quality inspector.",
) -> dict:
"""Create a vision fine-tuning example using base64-encoded image."""
with open(image_path, "rb") as f:
b64 = base64.b64encode(f.read()).decode("utf-8")
ext = Path(image_path).suffix.lstrip(".")
mime = {"jpg": "jpeg", "jpeg": "jpeg", "png": "png", "gif": "gif", "webp": "webp"}
data_url = f"data:image/{mime.get(ext, ext)};base64,{b64}"
return {
"messages": [
{"role": "system", "content": system_prompt},
{
"role": "user",
"content": [
{"type": "text", "text": instruction},
{"type": "image_url", "image_url": {"url": data_url}},
],
},
{"role": "assistant", "content": expected_response},
]
}
def build_vision_dataset(
labels_file: str, output_path: str, use_urls: bool = True
) -> None:
"""Build a JSONL dataset from a labels CSV.
Expected CSV format: image_path_or_url,instruction,expected_response
"""
import csv
examples = []
with open(labels_file, "r", encoding="utf-8") as f:
reader = csv.DictReader(f)
for row in reader:
if use_urls:
ex = create_vision_example_url(
row["image_url"], row["instruction"], row["expected_response"]
)
else:
ex = create_vision_example_base64(
row["image_path"], row["instruction"], row["expected_response"]
)
examples.append(ex)
with open(output_path, "w", encoding="utf-8") as f:
for ex in examples:
f.write(json.dumps(ex, ensure_ascii=False) + "\n")
print(f"Created {len(examples)} vision examples -> {output_path}")
if __name__ == "__main__":
build_vision_dataset("labels.csv", "vision_training.jsonl")
Tip: For vision fine-tuning, images must be accessible via URL (Azure Blob Storage with SAS token recommended) or encoded as base64 data URIs. Maximum image size and count limits apply – check the docs for current limits.
Distillation trains a smaller, cheaper model to replicate a larger model’s behavior on your specific task.
+-------------------+ generate +-------------------+
| Teacher Model |---responses---->| Training Data |
| (gpt-4o) | | (JSONL) |
+-------------------+ +---------+---------+
| fine-tune
v
+-------------------+
| Student Model |
| (gpt-4o-mini) |
+-------------------+
"""Distillation: use a large model to generate training data for a smaller model."""
import json
import os
import time
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2025-03-01-preview",
)
SYSTEM_PROMPT = "You are a legal contract summarizer. Summarize contracts concisely."
TEACHER_MODEL = "gpt-4o" # large, high-quality model
STUDENT_MODEL = "gpt-4o-mini-2024-07-18" # smaller, cheaper target
# -- Step 1: Generate training data with the teacher --
def generate_teacher_responses(
inputs: list[str], system_prompt: str = SYSTEM_PROMPT
) -> list[dict]:
"""Run inputs through the teacher model to generate gold-standard responses."""
examples = []
for user_input in inputs:
response = client.chat.completions.create(
model=TEACHER_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_input},
],
temperature=0.3, # low temp for consistent, high-quality outputs
)
assistant_response = response.choices[0].message.content
examples.append({
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_input},
{"role": "assistant", "content": assistant_response},
]
})
print(f" Generated response for: {user_input[:60]}...")
return examples
# Your task-specific inputs
contract_excerpts = [
"This Agreement is entered into as of January 1, 2025...",
"The Licensee shall pay a royalty of 5% on net sales...",
"Termination: Either party may terminate with 30 days notice...",
# ... load from your data source
]
teacher_data = generate_teacher_responses(contract_excerpts)
# Save training data
with open("distillation_train.jsonl", "w", encoding="utf-8") as f:
for ex in teacher_data:
f.write(json.dumps(ex, ensure_ascii=False) + "\n")
print(f"Generated {len(teacher_data)} training examples from teacher model.")
# -- Step 2: Fine-tune the student model --
training_file = client.files.create(
file=open("distillation_train.jsonl", "rb"),
purpose="fine-tune",
)
# Wait for processing
while client.files.retrieve(training_file.id).status != "processed":
time.sleep(5)
job = client.fine_tuning.jobs.create(
training_file=training_file.id,
model=STUDENT_MODEL,
hyperparameters={"n_epochs": 3},
suffix="contract-summarizer-distilled",
)
print(f"Distillation fine-tuning job: {job.id}")
# -- Step 3: Compare teacher vs student --
def compare_models(
test_input: str, teacher_model: str, student_deployment: str
) -> None:
"""Compare teacher and student outputs side by side."""
for label, model in [("Teacher", teacher_model), ("Student", student_deployment)]:
resp = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": test_input},
],
temperature=0.3,
)
print(f"\n-- {label} ({model}) --")
print(resp.choices[0].message.content)
print(f"Tokens: {resp.usage.total_tokens}")
| Metric | Teacher (gpt-4o) | Student (gpt-4o-mini distilled) |
|---|---|---|
| Quality | Baseline | ~90-95% of teacher |
| Latency | Higher | ~2-3x faster |
| Cost (inference) | ~$2.50/1M input | ~$0.15/1M input |
| Context window | 128k | 128k |
Track fine-tuning experiments with Weights & Biases for visualization, comparison, and reproducibility.
"""Integrate Azure OpenAI fine-tuning with Weights & Biases tracking."""
import os
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2025-03-01-preview",
)
# Create a fine-tuning job with W&B integration
job = client.fine_tuning.jobs.create(
training_file="file-abc123",
model="gpt-4o-mini-2024-07-18",
hyperparameters={"n_epochs": 3},
suffix="wandb-experiment",
integrations=[
{
"type": "wandb",
"wandb": {
"project": "azure-openai-fine-tuning",
"name": "contoso-support-v1",
"tags": ["gpt-4o-mini", "customer-support", "v1"],
},
}
],
)
print(f"Job with W&B: {job.id}")
# The training metrics (loss, accuracy) will automatically appear in your W&B dashboard.
# No additional code needed -- Azure OpenAI pushes metrics directly to W&B.
Prerequisites: Set your
WANDB_API_KEYenvironment variable or configure it in Azure AI Foundry settings. See W&B integration docs.
Before starting a fine-tuning project, answer these questions:
[ ] Have I tried prompt engineering thoroughly?
-> System messages, few-shot, chain-of-thought, structured output
[ ] Have I tried RAG?
-> If the model needs external knowledge, RAG is usually better
[ ] Do I have >= 50 high-quality examples?
-> Less than that rarely produces meaningful improvement
[ ] Is the task well-defined and consistent?
-> Vague tasks produce vague fine-tuned models
[ ] Am I willing to maintain the model?
-> Fine-tuned models need periodic retraining as data evolves
[ ] Does my budget allow hosting costs?
-> Fine-tuned models incur hourly hosting charges when deployed
| Scenario | Why Not | Better Alternative |
|---|---|---|
| Need up-to-date factual knowledge | FT bakes in training-time knowledge; can’t update easily | RAG with live data sources |
| Small, one-off task | ROI doesn’t justify the training + hosting cost | Prompt engineering |
| Highly variable output format | Hard to capture in training examples | Structured output / function calling |
| Need to cite sources | FT does not inherently ground responses | RAG with citation pipeline |
| Data changes daily/weekly | Constant retraining is expensive | RAG or prompt engineering |
| Regulatory requirement for explainability | FT is a black box | RAG with traceable retrieval |
| Want to add a single new capability | Overkill for one behavior | System prompt update |
| Anti-Pattern | Problem | Fix |
|---|---|---|
| Training on noisy / uncurated data | Model learns errors and inconsistencies | Curate and validate every example |
| Using too many epochs on small data | Severe overfitting; model memorizes examples | Use auto epochs or validate with a held-out set |
| Fine-tuning for factual recall | Model hallucinates “learned” facts | Use RAG for knowledge retrieval |
| Never evaluating the fine-tuned model | No idea if it is actually better than the base model | Run systematic evals before deploying |
| Skipping the validation file | Cannot detect overfitting during training | Always provide a validation split |
| Training on synthetic data without review | LLM-generated data may contain hallucinations | Human review all synthetic examples |
| One giant fine-tuning run | Hard to debug failures; expensive to redo | Iterate: small runs -> evaluate -> expand |
| Ignoring hosting costs | Surprise bills from deployed fine-tuned models | Budget for 24/7 hosting or deploy on-demand |
| Technique | Best For | Key Consideration |
|---|---|---|
| Continuous FT | Incremental improvement over time | Use lower LR, fewer epochs |
| Vision FT | Domain-specific image understanding | Requires image URLs or base64 |
| Distillation | Cost reduction with quality preservation | Teacher quality = student ceiling |
| W&B Integration | Experiment tracking and comparison | Set WANDB_API_KEY |
Next –> Knowledge Check