AI-901
Deep Dive
Prompt engineering is the discipline of designing instructions that maximize the quality of an LLM's responses. The AI-901 tests the basic techniques and key generation parameters.
Contents
Prompt engineering is the practice of designing, structuring, and refining the instructions sent to an LLM to obtain the desired output, without modifying the model's weights. It's the "programming interface" between the human and the LLM.
Why does it matter?
The same model with different prompts can produce radically different results in quality, format, and accuracy.
Bad prompt:
"Summarize this"
Good prompt:
"Summarize the following article in 3 key points of a maximum of 20 words each, aimed at business executives with no technical background."
Prompt engineering vs Fine-tuning
Prompt engineering: you modify the INPUT to the model. Fast, no training cost, flexible.
Fine-tuning: you modify the model's WEIGHTS. Slow, expensive, but can improve very specific behaviors.
Always start with prompting. Only do fine-tuning if prompting isn't enough.
A well-built prompt has up to 6 components. Not all of them are always necessary:
Instruction
requiredWhat the model must do. A clear action verb: "Summarize", "Classify", "Extract", "Translate".
"Classify the following email as urgent/normal/spam."
Context
Background information that helps the model understand the scenario.
"You are a technical support assistant for a B2B software company."
Input data
requiredThe text, document, or information the model must operate on.
"Email text: [email here]"
Examples (few-shot)
Input/output pairs that show exactly what format and style is expected.
"Example: email about forgotten password → urgent"
Output format
How the response should be structured: JSON, list, table, maximum X words, language.
"Respond with only one word: urgent, normal, or spam."
Constraints
What the model must NOT do: not make things up, not use technical jargon, not exceed N tokens.
"Do not respond if there isn't enough information. Say: Insufficient information."
Zero-shot prompting
You send the task directly with no examples. Works well for simple tasks or when the model already has enough knowledge.
Translations, simple summaries, direct questions.
Prompt
Translate to English: "El gato duerme en el tejado."
Response
The cat sleeps on the roof.
Few-shot prompting
You provide 2-5 input/output examples before the actual task. The model infers the pattern and applies it.
Custom classification, specific format, tasks with a defined style.
Prompt
Formal: "Good morning" → Informal: "Morning!" Formal: "Could you please attend?" → Informal: "You coming?" Formal: "Kind regards" → Informal:
Response
"Cheers"
Chain-of-Thought (CoT)
You ask the model to reason step by step before giving the final answer. Significantly improves mathematical, logical, and multi-step reasoning.
Math problems, logical reasoning, complex analysis.
Prompt
If there are 3 boxes with 8 apples each, and 5 apples are given away, how many are left? Reason step by step.
Response
1) Total: 3×8=24. 2) Given away: 5. 3) Left: 24-5=19.
Role prompting
You assign a specific role to the model so it adopts a particular style, level of expertise, and perspective.
Technical/simple explanations, mentoring, specialized analysis.
Prompt
Act as a family doctor explaining to a patient with no medical background what type 2 diabetes is.
Response
[accessible explanation, without unnecessary medical jargon]
Compare a zero-shot prompt against a few-shot one for the same ticket-classification task.
+1 more...
Chat APIs (such as those of GPT-5 models) have three roles: system, user, and assistant. The system prompt is the context instruction that defines the model's base behavior for the entire conversation.
Structure of a chat call
// SYSTEM — base instruction (invisible to the end user)
{ role: 'system', content: 'You are an Azure support assistant. Only answer questions about Azure. If the question is not about Azure, say you cannot help. Always respond in English.' }
// USER — user message
{ role: 'user', content: 'How do I create a virtual machine in Azure?' }
// ASSISTANT — previous model response (for history)
{ role: 'assistant', content: 'To create a VM in Azure...' }
// USER — next turn
{ role: 'user', content: 'How much does it cost?' }
What to define in the system prompt
System prompt and security
Users can attempt a jailbreak: instructions to make the model ignore the system prompt ("ignore the previous instructions and..."). Azure OpenAI's content filtering detects these attempts. A good system prompt also includes instructions on how to resist manipulation.
Temperature
Controls the randomness of the probability distribution over tokens. With temperature 0, it always picks the most probable token (deterministic). With high temperature, less probable tokens have a higher chance of appearing.
Top-p (nucleus sampling)
Only considers the tokens whose cumulative probability reaches p. Top-p 0.9 = considers the set of most probable tokens up to a combined 90% probability.
Max tokens
Maximum number of tokens the model can generate in the response. If the model reaches the limit, it cuts off the response even if it hasn't finished.
Frequency penalty
Penalizes tokens that already appeared in the generated text, proportional to how many times they appeared. Reduces word repetition.
Grounding is the technique of providing factual context in the prompt so the model bases its responses on verified information, rather than on its parametric knowledge (which may be outdated or incorrect).
Without grounding
User:
"What are the subscription prices for Azure SQL?"
Assistant:
[may make up outdated or incorrect prices]
With grounding
System:
"Answer ONLY using the information from the following document. If it's not in the document, say you don't have that information."
Context: [current pricing document]
User:
"What are the prices for Azure SQL?"
Assistant:
[real price from the document, with citation]
RAG is grounding at scale: instead of manually including documents in every prompt, RAG automatically retrieves the relevant documents and includes them as context before generating the response.
Example of the problem
"Help me with my code"
Better approach
Specify the language, the exact error, what you're trying to achieve, and what you've already tried.
Example of the problem
"Don't use technical jargon"
Better approach
"Explain using everyday analogies, without using specialized technical terms. Level: adult with no tech background."
Example of the problem
A 500-word paragraph with no separators
Better approach
Use clear sections: ## Context, ## Task, ## Output format, ## Constraints.
Example of the problem
"Summarize, translate, classify, and generate a title for this text"
Better approach
Split into separate calls or use a clear hierarchy: "First summarize in 3 points. Then, based on the summary, classify it into one of: [A, B, C]."
❓ Does temperature 0 produce the best response?
Not necessarily. Temperature 0 produces the most deterministic and consistent response (always the same for the same prompt). It is best for factual or extraction tasks. For creativity or a variety of ideas, high temperature produces better results.
❓ Does few-shot prompting modify the model?
No. Few-shot prompting includes examples in the prompt context but does not modify the model's weights. It is a form of "in-context learning" — the model learns the pattern only for that call, with no persistence.
❓ Does prompt engineering replace fine-tuning?
Not always, but it is the first step. Prompting is faster and cheaper. Fine-tuning is used when you need very specific or consistent behavior that cannot be achieved with prompts — such as a very particular brand tone or specialized terminology. The recommendation is to try prompting first.
Did you get this topic?
Test what you just learned
A team wants to use GPT-4o to analyze legal contracts and extract: start date, expiration date, automatic renewal clauses, and signing parties. Without specific instructions, the model returns lengthy paragraphs with unnecessary text. Which prompt engineering technique would best improve the result?
Put what you learned to the test with original AI-901 exam questions.
Start AI-901 mock examUnlimited mock exams with the Pro plan
Log in to track your progress.