Neural network architecture based on the "attention" mechanism, which processes entire sequences in parallel and weighs the relevance of each word relative to the others. It's the foundation of virtually all modern LLMs.
In AIF-C01 (Domain 1) and GENAI-L (Domain 1) it appears as a fundamental concept behind foundation models, without requiring mathematical depth. Question: which architecture enabled the quality leap in modern LLMs over earlier recurrent networks? Transformer.
Transformer architecture is a concept that can show up on the exam. Check the full guide to see it in context.