AI-901
Deep Dive
Domain D5 represents ~25% of the AI-901. It covers how generative models work, their limits, and how to work with them effectively through prompts and RAG.
Contents
Generative AI is the set of models capable of creating new content — text, images, code, audio, video — that did not previously exist in the training data. Unlike traditional ML (which predicts or classifies), GenAI generates.
✍️
Generates Text
ChatGPT, Claude, Gemini — writing, summaries, code, translation.
🎨
Generates Images
gpt-image, Stable Diffusion, Midjourney — image generation and editing.
💻
Generates Code
GitHub Copilot, GPT-5 — autocomplete, refactoring, test generation.
A Large Language Model (LLM) is a deep learning model trained on massive amounts of text to predict the next word (token) in a sequence. Through this simple objective, it learns grammar, facts, reasoning, and even code.
Pre-training
Trained on trillions of tokens from the internet, books, and code. The process requires thousands of GPUs over months.
Emergent capabilities
Capabilities that appear spontaneously as size increases: logical reasoning, arithmetic, analogies — without specific training.
In-context learning
The LLM can learn new tasks just from examples in the prompt, without retraining.
Parametric knowledge
Knowledge is encoded in the model's weights. It has no internet access unless explicitly integrated.
GPT-5 / GPT-5.x
OpenAI / Azure
Multimodal. Advanced reasoning. Available in Azure OpenAI.
GPT-5-mini / GPT-5-nano
OpenAI / Azure
Efficient and economical. Chat and high-volume production workloads.
Claude 3.x
Anthropic
Focused on safety and long context. Available outside Azure.
Llama 3
Meta (open)
Open-weights. Can be deployed on-premises or on Azure via the Model Catalog.
Phi-4
Microsoft
SLM (Small Language Model) — high capability with fewer parameters. On Azure.
gpt-image
OpenAI / Azure
Image generation from text. On Azure OpenAI.
All modern LLMs are based on the Transformer architecture (2017, "Attention Is All You Need"). Its key innovation is the attention mechanism, which allows the model to consider all the words in the context simultaneously instead of processing them sequentially.
Self-Attention
Each token "pays attention" to all other tokens in the context to understand relationships and dependencies.
Positional Encoding
Adds information about the position of each token in the sequence (order matters).
Feed-Forward Layers
Process information after attention. Store much of the model's "knowledge."
Layer Normalization
Stabilizes training and enables very deep models (100+ layers).
RNN/LSTM processed sequentially
→ Transformer processes in parallel → much faster
Long dependencies were "forgotten"
→ Attention maintains global context
Hard to scale to more GPUs
→ Parallelizable → scales to thousands of GPUs
Specialized models per task
→ A single foundation model for many tasks (transfer learning)
For the exam
You don't need to understand the math behind Transformers. You do need to know: (1) they are the foundation of modern LLMs, (2) they use attention mechanisms, (3) they benefit from parallelization.
LLMs don't process words — they process tokens. A token can be a whole word, part of a word, or a character. In English, 1 token ≈ 0.75 words. In other languages, the ratio can be lower.
// Tokenization example (GPT model tokenizer)
"ChatGPT is great!"
["Chat", "G", "PT", " is", " great", "!"]
// 6 tokens, 4 words
Why it matters
LLMs charge per token (input + output). The context limit is also measured in tokens. Text in other languages usually uses more tokens than English for the same content.
The context window is the maximum number of tokens the model can "see" in a single call — including the prompt, the conversation history, and the generated response.
Consequences of the context limit
If the conversation or document exceeds the limit, the model "forgets" the oldest parts. For long documents → RAG. For long conversations → summarize the history.
Prompt engineering is the practice of designing and structuring instructions (prompts) to get the best result from an LLM without modifying the model's weights.
Zero-shot prompting
You ask for the task directly without examples. Works well for simple tasks.
"Translate to English: El gato duerme."
Few-shot prompting
You provide 2-5 input/output examples before the actual task. Improves consistency on complex tasks.
"Formal: Good morning. Informal: Hey. → Formal: Please..."
Chain-of-thought (CoT)
You ask the model to reason step by step before giving the final answer. Improves mathematical and logical reasoning.
"Think step by step and then give me the answer."
System prompt
A context instruction that defines the role, response format and constraints. Sent before the conversation history.
"You are a technical assistant. Respond in English. Be concise."
Temperature
0.0 – 2.0Controls randomness. 0 = deterministic/repeatable; 2 = very creative/chaotic. For data extraction: low temperature. For creativity: high temperature.
Top-p (nucleus sampling)
0.0 – 1.0Considers only the tokens whose cumulative probability reaches p. 0.9 = considers the most probable 90%. An alternative to temperature.
Max tokens
integerLimit on the number of tokens the model can generate in the response. Important for cost and latency.
Stop sequences
stringsThe model stops generating when it encounters these sequences. Useful for structured formats.
Exam trap: temperature
Temperature 0 ≠ better quality. It means more determinism. For creativity (brainstorming, writing) → high temperature. For precise data extraction → low or 0 temperature.
RAG combines an LLM with an external knowledge base. Instead of relying solely on the model's knowledge (which may be outdated), RAG searches for relevant documents and includes them in the context before generating the response.
Indexing
Documents are split into chunks, converted into embeddings (numeric vectors) and stored in a vector database (e.g., Azure AI Search).
User query
The user's question is also converted into an embedding.
Retrieval
The chunks most semantically similar to the question are found using cosine similarity or other vector metrics.
Augmentation
The retrieved chunks are included in the prompt as additional context: "Based on these documents, answer...".
Generation
The LLM generates the response grounded in the retrieved documents, not just in its trained knowledge.
Advantages of RAG
When to use RAG
| Dimension | Fine-tuning | Prompting / RAG |
|---|---|---|
| What it modifies | The model's weights | Only the input to the model |
| Cost | High (GPU, time, data) | Low (only API tokens) |
| Data needed | Thousands of labeled examples | Few or no examples |
| Implementation time | Days/weeks | Hours |
| Best for | Very specific style, format, tone | New knowledge, personalization |
| Knowledge update | Requires retraining | Updates the index/documents |
Exam rule
If the question describes a need for new knowledge or recent data → RAG. If it describes a need for a very specific style or behavior (e.g., always responding with the brand's tone and jargon) → Fine-tuning. When in doubt, RAG first — it's faster and cheaper.
Hallucination is when an LLM generates incorrect information with total confidence — inventing facts, citations, people or data that don't exist. It's the most critical limitation for production deployments.
Hallucinations
Generates false information with apparent confidence. It cannot "know that it doesn't know."
Knowledge cutoff
Does not know about events after its training date. RAG mitigates this.
Context limit
Cannot process documents or conversations that exceed the context window.
Mathematical reasoning
Can fail at complex calculations. Mitigated with a code interpreter or external tools.
Bias
Inherits biases from the training data (internet text, books).
Non-determinism
With temperature > 0, the same question can produce different answers on each call.
Knowledge cutoff, factual hallucinations
Reduce variability on factual tasks
Ask the model to cite its sources
Block harmful or inappropriate responses
Human review for high-impact decisions
Critical facts verified by a separate system
Multimodal models can process and generate multiple types of data — text, images, audio, video — within a single architecture.
Text + Image (input)
GPT-5, Claude, Gemini — can "see" images and answer questions about them.
Analyzing screenshots, describing charts, reading scanned documents.
Text → Image (output)
gpt-image, Stable Diffusion, Imagen — generate images from textual descriptions.
Visual marketing, design prototypes, personalized illustrations.
Text + Audio
Models such as gpt-realtime and gpt-4o-transcribe transcribe, respond and generate audio.
Voice assistants, transcription + analysis, automated podcasts.
Video
Gemini 1.5 can analyze video. Sora generates video from text.
Video summaries, content moderation, clip generation.
❓ Does Generative AI always produce correct information?
No. LLMs can hallucinate — generate false information with apparent confidence. This is their best-known limitation. To reduce hallucinations: use RAG (ground responses in documents), lower the temperature, ask for source citations.
❓ Does RAG modify the model's weights?
No. RAG does not modify the model at all — it only enriches the prompt with retrieved documents before the model generates a response. Fine-tuning does modify the weights. RAG ≠ fine-tuning.
❓ Does high temperature improve accuracy?
No. High temperature = more creativity and variability. For tasks that require factual accuracy (data extraction, verifiable answers) → low or 0 temperature. For creativity (brainstorming, creative writing) → high temperature.
❓ Does an LLM know everything that has happened up to today?
No. LLMs have a knowledge cutoff date — they don't know about events after that date. Solution: RAG with up-to-date documents or integrated web search.
Did you get this topic?
Test what you just learned
A law firm has thousands of internal documents from past cases. They want a chatbot that answers lawyers' questions with accurate information, citing the source documents. The knowledge is updated with new cases every week. What is the best architecture?
Put what you learned to the test with original AI-901 exam questions.
Start AI-901 mock examUnlimited mock exams with the Pro plan
Log in to track your progress.