Topic 1 of 11
Choosing a compute option
The first line of section 2.1 of the exam guide is picking the right compute for each workload: Compute Engine, GKE, Cloud Run, Cloud Run functions, or Agent Runtime on Gemini Enterprise Agent Platform (formerly Vertex AI Agent Engine). It also asks when to use GPUs or TPUs. It all comes down to how much control you want over the infrastructure and how much you want Google to manage.
Contents
The official decision tree
Google Cloud's "Choose compute options" guide sums up the choice in five questions:
Do you want Google to manage all infrastructure? Yes → Cloud Run. No → question 2.
Do you want cluster orchestration (Kubernetes or Slurm)? Yes → question 4. No → question 3.
Do you want to manage VMs or bare metal instances yourself? Yes → Compute Engine. No → Batch.
Do you run standard containerized applications (microservices, APIs)? Yes → GKE. No → question 5.
Do you want Google to manage the cluster lifecycle? Yes → Cluster Director. No → Cluster Toolkit.
Compute Engine
Google Cloud's IaaS: self-managed VMs and bare metal instances, with a KVM hypervisor and Linux or Windows. Choose it when you need custom configurations, special kernel settings, or administrative control over the server.
- You control the OS, kernel, and boot files, and you choose processor, memory, storage, and guest OS.
- Orchestration is manual: you manage instances, network rules, routes, and backups. To scale, you add instances or use managed instance groups (MIGs).
- It supports every NVIDIA GPU and every TPU available.
- Machine families: general-purpose, compute-optimized, network-optimized, storage-optimized, memory-optimized, and accelerator-optimized. Some series allow custom machine types.
- Minimum uptime SLO of 99.5%, varying by region, network tier, and configuration.
Google Kubernetes Engine
A managed implementation of Kubernetes, which Google developed from its experience with Borg. It is ideal when you want to configure the infrastructure for your containers (networking, scaling, hardware, security) without managing the control plane.
Autopilot (recommended)
Google manages the control plane and nodes. You pay only for the resources your running Pods request. Pod-level SLA.
Standard
You manage the node pools. You pay for all node resources, whether Pods use them or not.
- Nodes are Compute Engine VMs grouped into a cluster; apps are packaged in containers deployed as Pods.
- It scales with the Horizontal and Vertical Pod Autoscalers (Pods) and the cluster autoscaler (nodes). Batch jobs are orchestrated with Kueue.
- It supports GPUs and TPUs for containers. There is a free tier for cluster management.
Cloud Run
A fully managed platform to run code, functions, or containers. Google manages the OS, kernel, networking, and capacity. You can deploy a container image or, for Go, Node.js, Python, Java, .NET, and Ruby, the source code directly.
| Resource | What for |
|---|---|
| Service | Responds to HTTP requests on a stable HTTPS endpoint (*.run.app) and also to events and functions |
| Job | Tasks that run to completion, manually or on a schedule. Array jobs start many instances in parallel |
| Worker pool | Always-on background workloads, such as consuming Kafka, Pub/Sub, or RabbitMQ queues |
| Instance | Long-lived workloads that need a singleton runtime environment |
- It scales to zero with no traffic; it can reach a thousand instances or more with a quota increase. Minimum instances avoid cold starts.
- Request-based billing (no charge without requests, plus a per-request fee) or instance-based billing (you pay for the instance's whole lifetime, no per-request fee). 100 ms granularity.
- The container file system is ephemeral: to persist data, use Cloud Storage or an NFS mount.
- It supports NVIDIA RTX PRO 6000 Blackwell and L4 GPUs. It doesn't support TPUs.
Cloud Run functions
Name change
Cloud Functions is now called Cloud Run functions. A function deployed to Cloud Run gets all of Cloud Run's benefits.
They are for event-driven services, lightweight APIs, and background tasks. Google manages the runtime and OS patching, and scales with traffic or events.
ETL and processing
React when a file is created, changed, or deleted in Cloud Storage.
Webhooks
HTTP trigger for events from GitHub, Slack, Stripe, or any system.
IoT
Process the data that thousands of devices send to Pub/Sub.
Connecting services
Logic between services: file uploads, log changes, Pub/Sub messages.
For backward compatibility with existing functions, gcloud functions deploy and the Cloud Functions v2 API still exist.
Agent Runtime on Gemini Enterprise Agent Platform
The exam guide names it "Agent Runtime on Gemini Enterprise Agent Platform (formerly Vertex AI Agent Engine)". It is a managed, serverless runtime to deploy, operate, and scale AI agents without managing infrastructure, with built-in observability.
- It supports agents built with Agent Development Kit (ADK), LangChain, LangGraph, LlamaIndex, Agent2Agent, AG2, or custom code.
- Related services: Sessions (conversation state), Memory Bank (long-term memory), Example Store and Evaluation Service, and sandboxed code execution.
- Security: IAM agent identity, service accounts, Agent Gateway, and Private Service Connect to reach your VPC.
- It is monitored with Cloud Trace, Cloud Logging, and Cloud Monitoring.
Choose it when the workload is an AI agent. For a regular API or website, Cloud Run is still the managed option.
Batch, VMware Engine, and AI clusters
| Option | When |
|---|---|
| Batch | Queued batch jobs (analysis, video conversion, simulations) without managing servers. It starts and stops VMs based on the queue. GPUs yes, TPUs no |
| Google Cloud VMware Engine | Migrate VMware workloads without changing applications. No GPUs or TPUs |
| Cluster Director | AI, ML, and HPC clusters with Slurm managed by Google |
| Cluster Toolkit | An open-source tool that deploys clusters from templates (blueprints) and leaves you in control of the OS |
Comparison
| Option | OS and kernel | Scaling | GPU / TPU | Billing |
|---|---|---|---|---|
| Compute Engine | Self-managed | Manual or with MIGs | All / all | By provisioned resources |
| GKE | Managed (Autopilot) or semi-managed (Standard) | Automatic Pods and nodes | All / all | By resources (Autopilot: by what Pods request) |
| Cloud Run | Google managed | Automatic by requests or CPU | RTX PRO 6000 and L4 / no | By use |
| Batch | Google managed | Based on the queue | Up to RTX PRO 6000 / no | By the VMs it starts |
GKE or Cloud Run
GKE
Complex microservices, stateful applications, workloads that need custom infrastructure or networking. You stay responsible for the Kubernetes aspects.
Cloud Run
Stateless web services and APIs, batch jobs, and workloads that benefit from pay-per-use pricing.
It isn't an either-or choice: both use standard container images (the same image works on both), declarative YAML APIs (the Cloud Run Admin API v1 is compatible with the Kubernetes API), Cloud Logging and Monitoring, Cloud Deploy, and load balancers that can split traffic between them for a gradual migration.
CPU, GPU, or TPU
TPUs (Tensor Processing Units) are Google-designed ASICs that accelerate ML, optimized for large matrix operations, with on-chip HBM memory, and groupable into slices. Code must be compiled with XLA. You use them from Compute Engine, GKE, and Gemini Enterprise Agent Platform; the Cloud TPU API is deprecated.
CPU
- Quick prototyping that needs maximum flexibility
- Simple or small models that train quickly
- Many custom TensorFlow operations written in C++
- Limited by host I/O or networking
GPU
- Many custom PyTorch or JAX operations that must run partly on CPUs
- TensorFlow ops not available on TPU
- Medium-to-large models with larger batch sizes
- Graphics, video transcoding, virtual desktops
TPU
- Models dominated by matrix computations
- No custom operations in the main training loop
- Training that runs for weeks or months
- Large models with large batches and ultra-large embeddings (ranking, recommendation)
When not to use TPUs
Programs with frequent branching or many element-wise operations, workloads that require high-precision arithmetic, custom operations in the training loop, and models with dynamically shaped tensors (XLA compiles for the first batch's shape).
| GPUs on Compute Engine | Use |
|---|---|
| A series (A4X Max, A4X, A4, A3, A2) | HPC, AI, and ML. Recent generations for pre-training and fine-tuning foundation models; A2 for smaller models and single-host inference. GPU included in the machine type |
| G series (G4, G2) | Graphics, simulation, transcoding, virtual desktops (NVIDIA RTX vWS). G2 uses NVIDIA L4 |
| N1 + GPU | You choose the GPU model and count when you create the VM (T4, P4, V100). Fewer GPUs cap the maximum vCPUs |
Exam scenarios
| Situation | Answer | Why |
|---|---|---|
| A legacy application needs a kernel with special parameters | Compute Engine | Full control of the OS, kernel, and boot. |
| A stateless API with traffic spikes and idle periods | Cloud Run | Scales to zero and charges for use. |
| Dozens of microservices with a service mesh and canary deployments | GKE | Kubernetes orchestration. |
| Process every image uploaded to a bucket | Cloud Run functions | A lightweight event-driven function. |
| Deploy an agent built with ADK or LangGraph to production without managing servers | Agent Runtime | A managed runtime for agents. |
| A queue of video conversion jobs without managing VMs | Batch | Google schedules and runs the jobs. |
| Migrate a VMware environment without changing applications | Google Cloud VMware Engine | Runs VMware VMs on Google hardware. |
| Train a large model dominated by matrix multiplications for weeks | TPU | Designed for large-scale matrix operations. |
Common exam traps
❓ Does Cloud Run support TPUs?
No. It supports NVIDIA RTX PRO 6000 Blackwell and L4 GPUs. TPUs are used from Compute Engine, GKE, or Gemini Enterprise Agent Platform.
❓ Do you pay per node in GKE Autopilot?
No. You pay for the resources your running Pods request. In Standard you pay for all node resources.
❓ Are Cloud Functions and Cloud Run functions different products?
No. Cloud Functions was renamed Cloud Run functions.
❓ Can you keep files on a Cloud Run container's disk?
Not persistently: the file system is ephemeral. Use Cloud Storage or NFS.
❓ Is a model with dynamically shaped tensors a good fit for TPUs?
No. XLA compiles for the first batch's shape, and recompiling on every change is too slow.
❓ Which GKE mode does Google recommend?
Autopilot, where Google also manages the nodes.
Did you get this topic?
Test what you just learned
A startup has a REST API in a container. Traffic is unpredictable and drops to zero at night. It doesn't want to manage clusters or pay when there are no requests. Which option should it choose?
Ready to test "Choosing a compute option"?
Original questions, each with a full explanation.
Start a mock exam