Brains4Cloud
ESEN
Practice

Topic 6 of 10

D3 · Ensuring the successful operation of a cloud solution

Operating databases

The second half of exam section 3.2 covers day-to-day data work: querying each database, estimating cost, backing up and restoring, reviewing BigQuery and Dataflow job status, and watching the whole fleet with Database Center. Choosing the service belongs to domain D2.

Querying each database

Almost every database has an editor built into the console, so you don't need a third-party client to run a query.

ServiceIn the consoleFrom the command line
Cloud SQL Studio: explorer, query editor, and results table; supports DDL, DML, and DQLgcloud sql connect opens the database client (mysql, psql…)
AlloyDB Studio: interactive SQL interface in the consolePostgreSQL client
Spanner Studio: explorer, query editor, and resultsgcloud spanner databases execute-sql
Bigtable Studio: no-code query builder and SQL editorcbt CLI (a gcloud component)
Firestore Studio: view, query (Query Builder tab), add, edit, and delete dataClient libraries
BigQuery Studio query editorbq query --use_legacy_sql=false
gcloud sql connect my-instance --user=root

gcloud spanner databases execute-sql example-database \
  --instance=example-instance --sql='SELECT * FROM MyTable WHERE MyKey = 1'

bq query --use_legacy_sql=false 'SELECT COUNT(*) FROM `my-project.sales.orders`'

gcloud components install cbt
cbt listinstances

The cbt CLI doesn't implement smart retries or error handling; for production Google recommends a Bigtable client library or the console. Its options (project, instance…) can be stored in a .cbtrc file.

A dry run validates the query without running it and returns an estimate of the bytes it would process (the on-demand cost). It uses no slots and is not charged. You can take the estimate to the pricing calculator.

bq query --use_legacy_sql=false --dry_run \
  'SELECT name FROM `my-project.sales.customers` WHERE country = "ES"'

When it isn't reliable

For federated queries on external sources it can report 0 bytes even though the real query is charged. On tables with row-level security it always returns 0 bytes.

Cloud SQL backups are incremental and encrypted with Google keys or CMEK. When you create a new instance, you choose one of two backup options:

Enhanced backups

Managed by Backup and DR Service in a centralized backup vault, through a backup plan. Retention of up to 10 years.

Standard backups

Managed by Cloud SQL. On-demand backups are kept until you delete them manually.

TypeHow it works
On-demandCreated at any time, for example before a risky operation
AutomatedScheduled. Google recommends not deleting them manually because point-in-time recovery needs them
FinalTaken when you delete the instance (can be made automatic). Kept for 30 days by default; 1 to 365 days for standard backups or up to 10 years for enhanced. Primary instances only, not replicas
gcloud sql backups create --async --instance=my-instance
gcloud sql backups list --instance=my-instance
gcloud sql backups restore BACKUP_ID --restore-instance=target-instance
  • Restoring onto an existing instance overwrites all of its data, including previous point-in-time recovery logs.
  • When you delete an instance, its on-demand and automated backups become retained backups at the project level, which you can restore to a new or existing instance.
  • Restoring from a backup doesn't keep the IP address of the deleted instance. To recover it with its IP, Cloud SQL keeps backups of deleted instances for 4 days, and within that window you can ask Customer Care to recover it exactly as it was.

Point-in-time recovery (PITR)

PITR restores a primary instance to its state at a specific moment, whether it is live or deleted. It always creates a new instance: you can't perform PITR onto an existing instance. In MySQL it requires binary logging and backups enabled, with continuous logs since the last backup before the incident. The recommended approach is a timestamp; in the console you use Create clone and Clone from an earlier point in time.

gcloud sql instances clone SOURCE_INSTANCE NEW_INSTANCE \
  --point-in-time='2026-10-24T08:15:00.000Z'

Events at or after that timestamp aren't reflected in the new instance.

Backups in Spanner, Firestore, AlloyDB, and Bigtable

  • Full backups on demand; with a schedule, full or incremental.
  • Retention of up to 1 year. The backup is stored in the same instance (and region) as the database; you can copy it to another region or project.
  • versionTime is the moment whose data the backup contains; createTime is when it was created. Scheduled backups start within 4 hours of versionTime.
gcloud spanner backups create my-backup --instance=my-instance \
  --database=my-db --retention-period=2w

  • Daily or weekly backup schedules, at most one of each per database. You can't choose the time.
  • Retention of up to 14 weeks. The backup is in the same location as the database.
  • You always restore to a new database. Deleting the database doesn't delete its backups.
  • Specific roles: roles/datastore.backupsAdmin, roles/datastore.backupSchedulesAdmin, roles/datastore.restoreAdmin…
gcloud firestore backups schedules create --database='(default)' \
  --recurrence=daily --retention=4w

gcloud firestore databases restore \
  --source-backup=projects/PROJECT_ID/locations/LOCATION/backups/BACKUP_ID \
  --destination-database=restored-db

  • Continuous backup and recovery is enabled by default on every cluster: it creates a new cluster from any recent state, with microsecond granularity. The window is 14 days by default, configurable from 1 to 35.
  • On-demand (full), automated (incremental), and system backups. They restore to a new cluster.
  • Like Cloud SQL, it offers enhanced (Backup and DR) or standard backups.

  • Standard backups (long retention) or hot backups (faster restore, for production).
  • Automated daily backup or on demand; they are incremental.
  • Retention of up to 90 days in the Enterprise edition or 365 days in Enterprise Plus (for example, 15d).
  • You restore to a new table, also in another instance.
gcloud bigtable backups create my-backup --instance=my-instance \
  --cluster=my-cluster --table=my-table --retention-period=7d

All Google Cloud data is encrypted at rest with AES-256 and Google keys, with nothing to configure. With CMEK you create the keys yourself in Cloud KMS: you control their rotation, access, and state, you see their usage logs, and you meet regulations that require your own keys. The resource's service agent handles encryption and decryption transparently.

  • Cloud KMS Autokey creates key rings and keys when the resource is created and grants the roles to the service agents. Google recommends it if its keys meet your requirements.
  • With Cloud SQL enhanced backups and CMEK, the Backup and DR service agent needs roles/cloudkms.cryptoKeyEncrypterDecrypter on the key.
  • Cloud KMS charges per key version and per cryptographic operation; default encryption is free.

BigQuery job status

Every load, export, query, or copy creates an asynchronous job. Its states are PENDING, RUNNING, and DONE; a finished job that failed includes errorResult. The cost is billed to the project that runs the job.

bq ls --jobs --filter="states:RUNNING,PENDING"
bq show --jobs JOB_ID
bq cancel JOB_ID
bq wait JOB_ID
RoleOn jobs
roles/bigquery.jobUserRun, list, view, and cancel their own
roles/bigquery.userThe same, but more permissive
roles/bigquery.resourceAdminList all jobs and view their metadata
roles/bigquery.adminList all jobs, view their metadata, and cancel any job

The console's monitoring interface shows the jobs list (running jobs and those from the last 30 days), the job graph with each step, execution details, metrics, estimated cost, recommendations, autoscaling charts, turnkey alerts for streaming, pipeline logs, and data sampling.

Cancel

Batch and streaming. Stops immediately, including buffered data: in-flight data can be lost. A cancelled job can't be restarted.

Drain

Streaming only. Stops ingesting new data and finishes processing its buffered data.

Force cancel

For jobs stuck in a regular cancel; you must try a regular cancel first.

gcloud dataflow jobs list --region=europe-west1
gcloud dataflow jobs describe JOB_ID --region=europe-west1
gcloud dataflow jobs drain JOB_ID --region=europe-west1
gcloud dataflow jobs cancel JOB_ID --region=europe-west1

Dataflow jobs can't be deleted, only archived; completed jobs are removed after 30 days. Before stopping a streaming pipeline, consider a snapshot to keep its state.

Database Center

An AI-assisted dashboard that gives you one centralized view of your whole database fleet, across projects. It uses data from your projects and Security Command Center to show health issues and recommendations, and it has a Gemini chat to ask about them.

  • It shows the inventory, the engines and versions in use, which critical databases aren't optimized for availability, which have no backups, which don't follow security best practices, and which aren't compliant with industry standards.
  • It monitors the health of AlloyDB, Bigtable, Cloud SQL (MySQL, PostgreSQL, and SQL Server), Firestore, Memorystore for Redis and Redis Cluster, Spanner, and Oracle Database@Google Cloud. For BigQuery it only shows inventory.
  • The data isn't real time: it usually updates within minutes, but can take up to 24 hours.

Exam scenarios

SituationAnswerWhy
Recover a Cloud SQL database to its state 2 hours ago, before a bad DELETEPITR with gcloud sql instances clone --point-in-timeAlways creates a new instance.
Find out how many bytes a BigQuery query will process without payingDry run (--dry_run)Uses no slots and is not charged.
Keep the data of a Cloud SQL instance that is about to be deletedFinal backup when deleting the instanceKept for 30 days by default.
Stop a streaming pipeline without losing the data it is already processingDrain the Dataflow jobStops reading and finishes its buffered data.
See which databases across the organization have no backupsDatabase CenterCentralized view of fleet health.
Automatic daily Firestore backups kept for 4 weeksgcloud firestore backups schedules createMaximum retention is 14 weeks.
Restore a Spanner backup in another regionCopy the backup to an instance in that regionA backup lives in the same instance as its database.
Run a SQL query on Spanner from the console without installing anythingSpanner StudioQuery editor built into the console.

Common exam traps

❓ Can you run Cloud SQL PITR onto the same instance?

No. Point-in-time recovery always creates a new instance (a clone).

❓ Does restoring a backup onto an existing instance keep its current data?

No. It overwrites all the data on the target instance.

❓ Does a BigQuery dry run cost money?

No. It validates the query and estimates bytes without using slots or charging.

❓ Can you drain a Dataflow batch job?

No. Drain is for streaming only; for batch you cancel.

❓ Does Database Center show data in real time?

No. It usually updates within minutes, but can take up to 24 hours.

❓ Can you restore a Firestore backup onto the original database?

No. You always restore to a new database.

Did you get this topic?

Test what you just learned

At 10:40 someone ran a DELETE without WHERE on a Cloud SQL for MySQL database that has automated backups and binary logging enabled. You need the data as it was at 10:39 without losing this morning's changes. What do you do?

Ready to test "Operating databases"?

Original questions, each with a full explanation.

Start a mock exam