AIF-C01
Deep Dive
Five concepts that appear in almost every generative AI question on the AIF-C01. They're the minimum technical vocabulary for understanding how a foundation model "thinks" and how it's configured.
Contents
A language model doesn't process text as characters or full words — it splits it into tokens: fragments that can be a whole word, part of a word, or even a single character or punctuation mark, depending on the tokenizer's vocabulary.
Practical example
The word "tokenization" might be split into several tokens like token, iza, tion, while very common words like "the" or "of" are usually a single token. On average, in English text, a token equals roughly ¾ of a word.
The cost of using a foundation model (and the use of the context window) is measured in tokens — both the ones you send in the prompt and the ones the model generates in the response.
Chunking consists of splitting long documents into smaller, more manageable fragments ("chunks") before generating their embeddings. It's an essential pre-step in RAG: instead of converting an entire 200-page document into a single, imprecise embedding, it's split into reasonably sized fragments (by paragraph, section, or a fixed number of tokens), each with its own embedding, which allows retrieving just the relevant part of the document instead of the entire document.
Why chunk size matters
Very small chunks lose context (an isolated sentence can be ambiguous); very large chunks dilute the precision of semantic search and consume more context window when retrieved. Chunk size is a design parameter that's tuned based on the type of document and the use case.
Multi-modal models
Multi-modal models can process and/or generate combinations of text, image, audio, or video in a single interaction, instead of being limited to a single input or output modality.
Diffusion models
Diffusion models are a type of model used mainly for image generation: they learn to gradually reverse a "noise" process added to an image, until producing a new, coherent image from random noise guided by a prompt.
An embedding is a numeric representation (a vector of hundreds or thousands of dimensions) of the meaning of a text, image, or other data. Two texts with similar meaning produce mathematically close vectors, even if they use completely different words.
Example
The embeddings for "the car broke down" and "the vehicle failed" end up very close in vector space, even though they don't share a single literal word — because their meaning is practically the same. This closeness is measured with metrics like cosine similarity.
Embeddings are the technical foundation of semantic search and RAG (Retrieval-Augmented Generation): instead of searching for exact word matches, you search for the documents whose embedding is closest to the embedding of the user's question.
Generate embeddings for three sentences in Bedrock and compare their semantic similarity, without them sharing a single word.
+1 more...
The context window is the maximum number of tokens (counting the input prompt plus the generated response) that a model can consider in a single interaction. Anything exceeding that limit simply isn't visible to the model.
Practical implications
The context window is just the physical limit of how much the model can process. Context engineering is the active practice of designing what specific information goes within that limit — which documents to include, which part of the conversation history to keep, which instructions to put first — to maximize response quality without wasting space on irrelevant information.
Context window vs. context engineering
The context window is a fixed property of the model (how many tokens fit). Context engineering is an application design decision: even with a huge context window, stuffing in irrelevant documents or an unnecessary conversation history dilutes the model's attention and increases cost — the goal is to carefully select what goes in, not simply fill all the available space.
At each step, a generative model computes a probability distribution over the next possible token. The inference parameters control how the final token is chosen from that distribution.
Temperature
Controls the overall randomness of the choice. With temperature = 0 the model always picks the most probable token (deterministic, repeatable responses). With temperature = 1 or higher, the probability of choosing less obvious tokens increases (more creative and varied responses, but less predictable).
Top-p (nucleus sampling)
Limits the choice to the smallest set of tokens whose cumulative probability reaches p. With top-p = 0.9, the model only considers the most probable tokens that together sum to 90% of the probability, discarding the "long tail" of unlikely options.
Top-k
Limits the choice to the k most probable tokens, regardless of their cumulative probability. With top-k = 10, the model chooses among the 10 highest-probability tokens at that step, discarding the rest entirely.
Combined numeric example
To build a customer-service chatbot that must give consistent, factual answers, use alow temperature (0.1-0.3). To generate creative marketing ideas, use a high temperature (0.7-1.0) combined with a wide top-p, to allow more variety in the responses.
Most foundation models (including those on Amazon Bedrock) are billed by tokens: one price per thousand or million input tokens (the prompt) and another price, usually higher, per output token (the generated response).
How this affects application design
A hallucination occurs when a model generates information that sounds convincing and coherent, but is objectively false or unsupported by any real source.
Why they happen
A foundation model doesn't "know" facts explicitly — it generates text by statistically predicting the most plausible next token given the context and its training patterns. If the model doesn't have enough reliable information about a topic, it will still generate a grammatically coherent response that appears certain, even if the factual content is made up.
Non-determinism — a different drawback
Unlike traditional software, the same prompt sent twice to the same model can produce different responses — this is generative AI's non-determinism. It happens because text generation is a probabilistic sampling process (affected by parameters like temperature and top-p), not an exact, repeatable calculation. It's a different limitation than hallucination: a response can be factually correct and still vary in its exact wording each time.
How they're mitigated (not eliminated)
Did you get this topic?
Test what you just learned
A team configures a legal assistant with Amazon Bedrock and needs the responses to be as consistent and predictable as possible, minimizing variability between runs of the same prompt. Which inference parameter setting is most appropriate?
Put what you learned to the test with original AIF-C01 exam questions.
Start AIF-C01 mock examUnlimited mock exams with the Pro plan
Log in to track your progress.