Gen AI Leader
Deep Dive
Anchoring a model's responses to real information is one of the most effective techniques for reducing hallucinations. Here we look at what grounding is, how RAG changes the generated output, Google Cloud's concrete offerings for achieving it, and the sampling parameters that control the model's final behavior.
Contents
Grounding is the practice of connecting a generative model's responses to verifiable data sources, instead of relying solely on the parametric knowledge the model memorized during training. This reduces hallucinations and increases the factual accuracy of responses. The exam guide distinguishes three types of data sources for grounding.
Proprietary enterprise data (first-party)
Documents, knowledge bases, internal policies, or operational data belonging to the organization itself. This is the most common type of grounding used in enterprise RAG solutions, because it gives full control over the source of truth.
Third-party data
Licensed or subscription-based external datasets — for example, market databases, supplier catalogs, or paid news feeds — integrated as an additional reference source.
World data
Public, up-to-date real-world information, typically obtained via real-time web search, useful for questions about recent events or general facts that go beyond the model's knowledge cutoff.
RAG (Retrieval-Augmented Generation) combines retrieval of relevant information from an external source (typically a vector database with embeddings) with generation from the language model: instead of answering solely from its memorized knowledge, the model receives the most relevant text fragments retrieved and uses them as additional context before generating the final response.
Concrete effect on the output
Google Cloud offers three distinct ways to implement grounding, with different levels of control versus ready-to-use management.
Agent Search (prebuilt RAG)
ManagedGoogle manages the ingestion, indexing, and retrieval of proprietary enterprise data. Use when you need fast grounding over your own data without building the RAG pipeline from scratch.
RAG APIs
Granular controlAPIs for building a custom RAG pipeline (chunking, embeddings, retrieval). Use when you need to customize how your own documents are processed, indexed, or retrieved.
Grounding with Google Search
World dataReal-time grounding using public web search results. Use when you need to answer questions about recent events, public facts, or information beyond the model's knowledge cutoff.
Watch out for this on the exam
Agent Search and RAG APIs are used for grounding with first-party or third-party data; Grounding with Google Search is the option when the required source of truth is the real world (world data), updated in real time. Choosing between them depends on what type of data source the use case needs, not on which one is "better" in general.
Besides grounding, a generative model's behavior is controlled with inference parameters — the exam guide expects you to recognize five: token count, temperature, top-p (nucleus sampling), safety settings, and output length.
Temperature
Controls the randomness of the next-token choice. With temperature = 0 the model always picks the most probable token (deterministic, repeatable responses). With higher values (e.g. 0.8-1.0) the probability of picking less obvious tokens increases, giving more varied and creative but less predictable responses.
Top-p (nucleus sampling)
Limits the choice to the smallest set of tokens whose cumulative probability reaches the threshold p. With top-p = 0.9, the model only considers the most probable tokens that together add up to 90% probability, discarding the "long tail" of unlikely options — a distinct and complementary control to temperature, not a substitute for it.
Token count
The number of tokens consumed by the input prompt plus the generated response. It directly affects the cost per request and how much context fits within the model's context window.
Safety settings
Configurable thresholds for blocking or filtering potentially harmful content (violence, sexual content, hate speech, among other categories) before the response reaches the end user.
Output length
The maximum number of tokens the model can generate in its response. Setting it correctly avoids responses cut off mid-sentence or unnecessarily long responses that increase cost.
How they combine in practice
A typical case: a customer service chatbot sets a low temperature for consistent responses, strict safety settings to avoid inappropriate content, and a bounded output length to keep responses short and control cost — all of this is tuned together with the chosen grounding strategy, not in isolation.
Did you get this topic?
Test what you just learned
A company needs its gen AI assistant to answer customer questions citing exclusively the company's own internal return policies, without using external or internet information. Which Google Cloud grounding offering is the most suitable starting point?
Put what you learned to the test with original Gen AI Leader exam questions.
Start Gen AI Leader mock examUnlimited mock exams with the Pro plan
Log in to track your progress.