AIF-C01
Deep Dive
Foundation models are the heart of modern generative AI and the central axis of domain D2 of the exam (24% of the total weight). Understanding what makes them different from traditional ML is the foundation for everything else: Bedrock, prompting, RAG, and fine-tuning.
Contents
A foundation model is a deep learning model trained with massive amounts of diverse, unlabeled data, designed to serve as a general base adaptable to many different tasks — instead of solving a single specific problem the way traditional ML did.
The same text foundation model can, without retraining, answer questions, summarize documents, translate, write code, or hold a conversation — simply by changing the prompt it receives.
Traditional ML — single task
A model trained to classify spam only does that. A model trained to predict housing prices only does that. Each new task requires training (or at least significantly retraining) a new model, with its own labeled dataset specific to that task.
Foundation model — multi-task
A single model, trained once with massive, general data, adapts to dozens of different tasks simply through the prompt (natural-language instruction), without retraining. The generality comes from the scale of the pre-training, not from training for each task individually.
For the exam
If a scenario describes "a single model that answers questions, summarizes text, AND generates code without retraining for each task", it's describing a foundation model. If it describes "a model trained specifically to detect fraud with labeled historical fraud data", it's traditional ML.
Foundation models are initially trained ("pre-trained") with enormous volumes of unlabeled data — internet text, books, code, images — using self-supervised learning tasks, such as predicting the next word of a sentence given the preceding context. It doesn't require a human to label each example: the text's own structure serves as the training signal.
This pre-training is extremely expensive in compute (weeks of training on massive GPU clusters) and only a few organizations do it. The result is a base model with a broad understanding of language, which can then be adapted to specific tasks with much less effort via prompt engineering, RAG, or fine-tuning.
Transfer learning is the technique that allows the general knowledge acquired during a foundation model's pre-training to be reused for specific tasks, without having to train from scratch. It's the economic reason foundation models are viable: training from scratch costs millions of dollars; adapting a pre-trained model costs a fraction of that.
Example
A foundation model pre-trained on general internet text already "knows" grammar, general facts, and basic reasoning. Fine-tuning it with a small dataset of a company's technical support tickets transfers that general knowledge to the specific domain, requiring far less data and compute than training a language model from scratch.
Text (LLMs)
Generate, summarize, translate, and reason over natural language. Examples: Claude, Amazon Nova, Llama.
Image
Generate images from text descriptions (text-to-image) or transform existing images. Example: Amazon Nova Canvas, Stable Image (Stability AI).
Multimodal
Process and/or generate combinations of text, image, audio, or video in a single interaction. Example: Claude with vision capability.
Code
Specialized in generating, explaining, and completing source code across multiple programming languages. Example: the models behind Kiro (the successor to Amazon Q Developer).
Send an image and a text prompt to a multimodal Bedrock model to describe and assess damage.
+1 more...
From the decision to build or adapt a foundation model to it being in production, the process follows a cycle with well-defined stages — and it ends in a continuous feedback loop, not a single final step.
1. Data selection: Training data sources are identified and prepared — text, code, images — evaluating their quality, diversity, and potential biases.
2. Model selection: The architecture and model size suited to the compute budget and the goal are chosen (or an existing foundation model is chosen as a base instead of training from scratch).
3. Pre-training: The model learns general language/data patterns with self-supervised learning over the massive dataset, in an extremely compute-expensive process.
4. Fine-tuning: The base model is adapted to specific tasks or domains with smaller, (usually) labeled datasets, leveraging transfer learning.
5. Evaluation: The model's performance is measured with relevant metrics and benchmarks before deploying it, verifying accuracy, bias, and edge-case behavior.
6. Deployment: The model goes into production, exposed through an API (for example, via Amazon Bedrock) for applications to consume.
7. Feedback (continuous loop): The model's production behavior is monitored and real usage feedback is collected, which feeds new rounds of fine-tuning, data adjustment, or even retraining — the cycle doesn't end at deployment, it feeds back continuously.
For the exam
The key point is that the cycle is iterative, not a single linear pass: production feedback feeds back into earlier stages (more data, more fine-tuning), instead of the process ending once the model is deployed.
Did you get this topic?
Test what you just learned
What is the main difference between a foundation model and a traditional ML model trained for a specific task?
Put what you learned to the test with original AIF-C01 exam questions.
Start AIF-C01 mock examUnlimited mock exams with the Pro plan
Log in to track your progress.