This lesson covers the core techniques for getting reliable, high-quality outputs from language models — from basic zero-shot prompting to advanced patterns like chain-of-thought, structured outputs, and function calling.
Official Documentation: Advanced Prompt Engineering
Zero-shot prompting provides instructions without any examples. The model relies on its training to interpret the task.
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint="https://your-resource.openai.azure.com/",
api_key="your-api-key",
api_version="2024-12-01-preview"
)
# Zero-shot classification
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{
"role": "system",
"content": "Classify the following customer message into one of these categories: "
"billing, technical_support, account, feedback, other. "
"Respond with only the category name."
},
{
"role": "user",
"content": "I was charged twice for my subscription last month."
}
],
temperature=0.0
)
print(response.choices[0].message.content)
# Output: billing
When to use zero-shot:
Few-shot prompting provides examples of input/output pairs to guide the model’s format and behavior.
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{
"role": "system",
"content": "Extract structured product information from descriptions."
},
# Example 1
{
"role": "user",
"content": "The new UltraBook Pro 15 weighs just 1.2kg and has 32GB RAM with a 1TB SSD."
},
{
"role": "assistant",
"content": '{"name": "UltraBook Pro 15", "weight_kg": 1.2, "ram_gb": 32, "storage": "1TB SSD"}'
},
# Example 2
{
"role": "user",
"content": "CloudPhone X features a 6.7 inch OLED display and 256GB storage at $899."
},
{
"role": "assistant",
"content": '{"name": "CloudPhone X", "display": "6.7 inch OLED", "storage": "256GB", "price": "$899"}'
},
# Actual request
{
"role": "user",
"content": "The SpeedRunner 3000 gaming laptop has an RTX 4090, 64GB DDR5, and costs $2,499."
}
],
temperature=0.0
)
print(response.choices[0].message.content)
# Output: {"name": "SpeedRunner 3000", "gpu": "RTX 4090", "ram_gb": 64, ...}
Few-shot best practices:
Chain-of-thought prompting asks the model to reason step-by-step before providing a final answer, improving accuracy on complex reasoning tasks.
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{
"role": "system",
"content": "You are a math tutor. Think through problems step by step "
"before giving the final answer."
},
{
"role": "user",
"content": (
"A store sells notebooks for $3 each. If you buy 5 or more, you get "
"a 20% discount on the entire order. You also have a $2 coupon. "
"How much do you pay for 7 notebooks?"
)
}
],
temperature=0.0
)
print(response.choices[0].message.content)
# The model will show its reasoning:
# Step 1: 7 notebooks × $3 = $21
# Step 2: 20% discount: $21 × 0.80 = $16.80
# Step 3: Apply $2 coupon: $16.80 - $2 = $14.80
# Final answer: $14.80
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{
"role": "system",
"content": (
"You are an expert analyst. For each question:\n"
"1. State your REASONING in a <thinking> section\n"
"2. State your ANSWER in an <answer> section\n"
"Be thorough in your reasoning but concise in your answer."
)
},
{
"role": "user",
"content": "Should we migrate our on-premises SQL Server to Azure SQL "
"Database or Azure SQL Managed Instance? We have 50 databases, "
"use cross-database queries, SQL Agent jobs, and linked servers."
}
],
temperature=0.3
)
| Component | Purpose | Example |
|---|---|---|
| Capacity | Who is the AI? | “You are a senior DevOps engineer” |
| Role | What is its job? | “Your job is to review infrastructure code” |
| Instructions | What are the rules? | “Check for security issues, cost waste, and best practices” |
| Standard | What format? | “Output a markdown table with severity, issue, and fix” |
| Personality | What tone? | “Be direct and technical, no fluff” |
| Experiment | Iterate | Test and refine based on outputs |
SYSTEM_PROMPT = """You are a customer support agent for Contoso Electronics.
ROLE: Help customers with product questions, order status, returns, and troubleshooting.
RULES:
- Only answer questions related to Contoso products and services.
- If the customer asks about something you don't know, say: "Let me connect you with a specialist."
- Never make up product specifications, prices, or policies.
- Do not discuss competitors or provide opinions on non-Contoso products.
- If the customer is frustrated, acknowledge their feelings before providing solutions.
OUTPUT FORMAT:
- Keep responses under 150 words unless the customer asks for detail.
- Use bullet points for multi-step instructions.
- Include relevant product links when available.
TONE: Friendly, professional, helpful. Never condescending."""
Force the model to output valid JSON:
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{
"role": "system",
"content": "Extract order information as JSON with fields: "
"order_id, product, quantity, total_price, status."
},
{
"role": "user",
"content": "Order #45892 for 3 wireless keyboards at $29.99 each "
"is currently being shipped. Total: $89.97."
}
],
response_format={"type": "json_object"},
temperature=0.0
)
import json
data = json.loads(response.choices[0].message.content)
print(json.dumps(data, indent=2))
# {
# "order_id": "45892",
# "product": "wireless keyboard",
# "quantity": 3,
# "total_price": 89.97,
# "status": "shipped"
# }
For strict schema adherence, use response_format with a JSON schema:
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{
"role": "system",
"content": "Analyze the sentiment of the given text."
},
{
"role": "user",
"content": "The new update is fantastic! Loading times are way faster."
}
],
response_format={
"type": "json_schema",
"json_schema": {
"name": "sentiment_analysis",
"strict": True,
"schema": {
"type": "object",
"properties": {
"sentiment": {
"type": "string",
"enum": ["positive", "negative", "neutral", "mixed"]
},
"confidence": {
"type": "number",
"description": "Confidence score between 0 and 1"
},
"key_phrases": {
"type": "array",
"items": {"type": "string"}
}
},
"required": ["sentiment", "confidence", "key_phrases"],
"additionalProperties": False
}
}
},
temperature=0.0
)
💡 Tip: Always mention “JSON” in the system or user message when using JSON mode — the model needs this hint to produce valid JSON.
Function calling lets the model decide when to invoke external tools and structures the arguments for you.
import json
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g. 'Seattle, WA'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit"
}
},
"required": ["city"]
}
}
},
{
"type": "function",
"function": {
"name": "search_products",
"description": "Search the product catalog by name or category",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Search query"},
"category": {"type": "string", "description": "Product category filter"},
"max_results": {"type": "integer", "description": "Max results (default 5)"}
},
"required": ["query"]
}
}
}
]
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{"role": "system", "content": "You are a helpful assistant with access to tools."},
{"role": "user", "content": "What's the weather like in Seattle?"}
],
tools=tools,
tool_choice="auto"
)
# Check if the model wants to call a function
message = response.choices[0].message
if message.tool_calls:
for tool_call in message.tool_calls:
func_name = tool_call.function.name
func_args = json.loads(tool_call.function.arguments)
print(f"Model wants to call: {func_name}({func_args})")
# After executing the function, send the result back
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What's the weather like in Seattle?"},
message, # The assistant message with tool_calls
{
"role": "tool",
"tool_call_id": message.tool_calls[0].id,
"content": json.dumps({
"city": "Seattle",
"temp": 62,
"unit": "fahrenheit",
"condition": "cloudy"
})
}
]
final_response = client.chat.completions.create(
model="gpt-4o-prod",
messages=messages,
tools=tools
)
print(final_response.choices[0].message.content)
# "The current weather in Seattle is 62°F and cloudy."
Both control randomness, but they work differently:
| Temperature | Top P | |
|---|---|---|
| How | Scales the probability distribution | Truncates to top cumulative probability |
| Low value | More deterministic, repetitive | Fewer token candidates |
| High value | More random, creative | More token candidates |
| Recommended | Adjust this for most use cases | Keep at 0.95 unless you have a reason |
⚠️ Don’t adjust both simultaneously. Change one at a time. In most cases, adjusting
temperaturealone is sufficient.
# Recipe 1: Factual extraction (classification, NER, data extraction)
FACTUAL = {"temperature": 0.0, "top_p": 1.0, "max_tokens": 500}
# Recipe 2: Conversational (chatbot, Q&A)
CONVERSATIONAL = {"temperature": 0.7, "top_p": 0.95, "max_tokens": 800}
# Recipe 3: Creative writing (stories, marketing copy)
CREATIVE = {"temperature": 1.0, "top_p": 0.9, "max_tokens": 2000,
"presence_penalty": 0.6}
# Recipe 4: Code generation
CODE_GEN = {"temperature": 0.2, "top_p": 0.95, "max_tokens": 4000}
# Recipe 5: Brainstorming (many diverse ideas)
BRAINSTORM = {"temperature": 1.2, "top_p": 0.85, "presence_penalty": 0.8,
"frequency_penalty": 0.5}
| Penalty | Effect | Use Case |
|---|---|---|
| Frequency penalty | Reduces probability of tokens proportional to how many times they appeared | Reduce word-for-word repetition |
| Presence penalty | Reduces probability of any token that appeared at all (flat penalty) | Encourage topic diversity |
# Example: Generate diverse product taglines
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{"role": "system", "content": "Generate 5 unique marketing taglines."},
{"role": "user", "content": "Product: Eco-friendly water bottle made from recycled ocean plastic."}
],
temperature=0.9,
presence_penalty=0.6, # Encourage diverse vocabulary
frequency_penalty=0.3, # Reduce repetitive phrasing
max_tokens=300
)
Break complex tasks into sequential steps:
def analyze_and_recommend(client, article_text):
"""Two-step chain: summarize, then recommend actions."""
# Step 1: Summarize
summary_response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{"role": "system", "content": "Summarize the article in 3 bullet points."},
{"role": "user", "content": article_text}
],
temperature=0.3,
max_tokens=200
)
summary = summary_response.choices[0].message.content
# Step 2: Generate recommendations based on the summary
rec_response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{"role": "system", "content": "Based on this summary, provide 3 actionable recommendations."},
{"role": "user", "content": summary}
],
temperature=0.5,
max_tokens=300
)
return {
"summary": summary,
"recommendations": rec_response.choices[0].message.content
}
Generate multiple responses and take the majority answer for higher accuracy:
def classify_with_consistency(client, text, n_samples=5):
"""Run classification multiple times and take majority vote."""
results = []
for _ in range(n_samples):
response = client.chat.completions.create(
model="gpt-4o-prod",
messages=[
{"role": "system", "content": "Classify as: positive, negative, or neutral. Reply with one word."},
{"role": "user", "content": text}
],
temperature=0.7,
max_tokens=10
)
results.append(response.choices[0].message.content.strip().lower())
from collections import Counter
majority = Counter(results).most_common(1)[0]
return {"label": majority[0], "confidence": majority[1] / n_samples, "votes": results}
result = classify_with_consistency(client, "The product works but the delivery was slow.")
print(result)
# {'label': 'neutral', 'confidence': 0.6, 'votes': ['neutral', 'neutral', 'neutral', 'negative', 'negative']}
💡 Be specific about what you DON’T want. Telling the model “Do not include disclaimers” or “Do not add explanations” is often as important as saying what to include.
💡 Use delimiters for separating instructions from content: triple backticks, XML tags, or markdown headers. This prevents prompt injection.
💡 Test your prompt at temperature 0 first. Once the core behavior is correct, increase temperature for variety if needed.
💡 JSON mode requires the word “JSON” in your prompt. Without it, the model may not output valid JSON even with
response_formatset.
💡 Function calling is more reliable than asking the model to output function call syntax in text. Always use the
toolsparameter for tool use.
Next: Lesson 3 — Prompt Flow