Chapter 17

Logging and Audit

Scope. This chapter covers what Google Cloud records about itself, where those records go, how long they survive, and who can change them. It owns audit log types, the Logs Router, buckets, views, sinks, retention, immutability, export destinations, and the cost model. The design argument for a dedicated logging project is §2.22; alerting on log content is §18.15; Security Command Center's own finding export is §16.15; evidence handling during an incident is §34.13. Prerequisites. Chapter 2 (§2.22 logging projects, §2.30 organization policy), Chapter 3 (§3.9 roles, §3.17 deny policies), Chapter 14 (§14.8 key management). Verified against. Google Cloud console and API surface as of 2026-09, Cloud SDK 583.0.0, hashicorp/google provider 8.x; see sources at end.

Audit logging is the control that makes every other control provable. Without it, "the bucket was private" is an assertion; with it, it is a record with a timestamp, a principal, and a request. That is why this chapter sits between detection (Chapter 16) and monitoring (Chapter 18) rather than in an operations appendix: the log is not operational telemetry that happens to be useful for security, it is the primary security artifact, and Google generates most of it whether you ask or not.

The design problem is that Google's defaults are shaped for cost, not for investigation. Admin Activity logs are always on and free of ingestion charge; Data Access logs — the ones that record who read what — are off by default in almost every service, because turning them on in a busy estate produces terabytes. Every organization that has ever tried to answer "did the attacker read the customer table?" after the fact has discovered which of those two categories they were relying on.

The second problem is jurisdictional. Logs are written into the project where the event happened, and the person who administers that project can change what is stored there. An audit trail an administrator can edit is not an audit trail. Solving that requires routing logs out of the project as they are produced (§17.9), into a bucket a different team controls (§17.12), locked so that even that team cannot shorten its retention (§17.14). The rest of this chapter is the mechanics of getting there, and §17.20 is the honest accounting of what that design still does not protect against.

17.1 Cloud Audit Logs §

Cloud Audit Logs are the records Google Cloud writes about administrative and data operations against your resources. Google states plainly that "Cloud Audit Logs provides four types: Admin Activity, Data Access, System Event, and Policy Denied."

The four types differ in what generates them and whether you control them:

Every audit log entry is a protoPayload of type AuditLog, and four of its fields carry almost all investigative value: authenticationInfo.principalEmail (who), methodName (what), resourceName (to which resource), and requestMetadata.callerIp (from where). A log query that does not project those four fields is usually asking the wrong question.

Audit logs are addressed by a reserved log ID, which is what makes them filterable independently of application logs:

gcloud logging read \
  'logName:"logs/cloudaudit.googleapis.com%2Factivity"
   AND protoPayload.authenticationInfo.principalEmail:"@rickcollette.domain"' \
  --project=rc-saas-prod-app-01 --freshness=7d --limit=50

Console. Console → Logging → Logs Explorer, with the Log name facet showing activity, data_access, system_event, and policy under cloudaudit.googleapis.com.

Judgment. Treat the two free, always-on types as your baseline evidence and the two billable types as deliberate investments. §17.3 covers how to make the Data Access investment without buying the whole estate's read traffic.

Pitfall. Audit logs record the API call, not its effect. A SetIamPolicy entry tells you the policy was replaced and by whom; it does not, on its own, tell you what the policy said before. Reconstructing prior state needs either the request payload (present only when Data Access logging captures it) or Cloud Asset Inventory history (§29.6).

17.2 Admin Activity Logs §

Admin Activity audit logs record every API call that modifies the configuration or metadata of a resource — creating a VM, changing an IAM policy, deleting a bucket.

They are the one log you cannot lose. Google's statement is unambiguous: "Admin Activity audit logs are always written; you can't configure, exclude, or disable them." They are also not charged for ingestion, and a subset of them is routed to the _Required bucket where it survives for 400 days regardless of what anyone configures (§17.13).

This is the log that answers the questions that matter most after a compromise:

• Which principal granted the role that the attacker used?
• When was the firewall rule created, and from what IP was the API call made?
• Who disabled the organization policy constraint?
• Which service account created the instance the miner ran on?

The permission to read them is logging.privateLogViewer or logging.viewer, and the distinction matters: roles/logging.viewer reads ordinary logs, while roles/logging.privateLogViewer is required for Data Access logs (§17.3). Grant the narrower role by default.

Judgment. Because Admin Activity logs cannot be disabled, they are the correct basis for the highest-severity alerts (§18.15) — an alert built on them cannot be evaded by turning logging off. Every alert in this book that fires on a control-plane change is written against this log.

Pitfall. "Always written" is not "always retained where you can see it". The full stream lands in the _Default bucket with a 30-day retention unless you route it elsewhere. Only the subset Google places in _Required gets 400 days automatically, and you do not choose what is in that subset.

17.3 Data Access Logs §

Data Access audit logs record reads of resource configuration or metadata, and reads and writes of user data. They are the difference between knowing an attacker had access and knowing what they took.

They are off by default and Google says why: "Except for BigQuery, Data Access audit logs are disabled by default because they can generate large volumes of data."

Three log types are configured independently, which is what makes selective enablement practical:

Configuration is an IAM policy field, not an organization policy. It lives in the auditConfigs block of the resource's IAM policy, and it is inherited: "You can set a configuration that all new and existing Google Cloud services in your Google Cloud project, folder, or organization inherit."

auditConfigs:
  - service: allServices
    auditLogConfigs:
      - logType: ADMIN_READ
      - logType: DATA_WRITE
  - service: storage.googleapis.com
    auditLogConfigs:
      - logType: DATA_READ
      - logType: DATA_WRITE

Enable by service, not by allServices, wherever volume is a concern. The pattern above is this book's default: ADMIN_READ and DATA_WRITE everywhere, plus DATA_READ only on the services that hold regulated data — Cloud Storage, BigQuery, Cloud SQL, Secret Manager. DATA_READ on allServices in a busy estate is the single largest logging bill most organizations ever generate.

Exempted principals are a real risk. "When you add an exempted principal, audit logs aren't created for them for the selected log types" — a service account exempted for cost reasons is a service account whose data reads are invisible. Prevent this with constraints/iam.disableAuditLoggingExemption at the organization.

Pitfall. Several Event Threat Detection detectors (§16.7), including the BigQuery exfiltration rule, reason over Data Access logs. Leaving them off does not produce an error or an empty result — it produces a detector that never fires, which is indistinguishable from safety.

17.4 System Event Logs §

System Event audit logs record administrative actions that Google Cloud takes on your resources without an API call from you: a live migration, an automatic instance restart, a preemption, a Google-initiated maintenance action.

They are always on and cannot be suppressed — "System Event audit logs are always written; you can't configure, exclude, or disable them" — and, like Admin Activity logs, they carry no ingestion charge.

Their security value is negative evidence. When an instance disappears or a disk is detached, the System Event log distinguishes a platform action from a human one. That distinction is the first branch of most VM incident triage (§34.7): if the event is in system_event, the answer is a maintenance event; if it is in activity, someone did it.

They are addressed by their own log ID:

gcloud logging read \
  'logName:"logs/cloudaudit.googleapis.com%2Fsystem_event"
   AND resource.type="gce_instance"' \
  --project=rc-saas-prod-app-01 --freshness=30d --limit=20

Pitfall. Because nobody configures them, System Event logs are routinely excluded from the sink filters teams write by hand — a filter of logName:"activity" OR logName:"data_access" silently drops them. Write sink filters that include the whole cloudaudit.googleapis.com namespace and exclude specific noise, rather than enumerating the types you remembered.

17.5 Policy Denied Logs §

Policy Denied audit logs record requests that a Google Cloud security policy rejected — most importantly VPC Service Controls violations (Chapter 20) and context-aware access denials.

They cannot be disabled but they can be excluded, and they are billed. Google's wording carries all three facts: "Policy Denied audit logs are generated by default and your Google Cloud project is charged for log storage. You can't disable Policy Denied audit logs, but you can use exclusion filters to prevent Policy Denied audit logs from being stored in Cloud Logging."

This is the log that makes a perimeter operable. A VPC Service Controls perimeter in dry-run mode (§20.8) produces nothing else — the denial that would have happened appears here and nowhere else, which is why perimeter rollout is a log-reading exercise rather than a testing exercise.

Never exclude them from the security sink. The exclusion Google offers exists for cost control in projects generating high volumes of expected denials. Excluding them from the _Default sink to save money is defensible; excluding them from the aggregated sink that feeds your perimeter analysis destroys the only record of what the perimeter is doing.

Pitfall. A noisy Policy Denied stream is usually a misconfigured perimeter, not a logging problem. Teams that respond to the volume with an exclusion filter make the perimeter permanently unobservable and then enforce it anyway, which is how a perimeter takes down production.

17.6 Cloud Logging §

Cloud Logging is the ingestion, routing, storage, and query service that every log in Google Cloud passes through. Its architecture is one pipeline with one decision point.

Everything flows through the Logs Router. A log entry arrives, the Router evaluates every sink in scope against it, and each matching sink copies the entry to its destination. Nothing is stored until a sink stores it.

Two sinks exist in every project before you do anything. The _Required sink "routes a subset of audit logs to the resource's _Required log bucket" and "You can't modify or delete the _Required log sink." The _Default sink "routes log entries to the resource's _Default log bucket", and — critically — "You can modify and disable the _Default log sink."

The Router is not retroactive. "Log sinks can't retroactively route log entries. That is, a log sink can't route a log entry that was received before the sink was created." Every hour a sink does not exist is an hour of evidence that exists only in the _Default bucket, on a 30-day clock.

Judgment. Create the organization-level aggregated sink first, before any workload project exists. It is the one piece of this chapter that cannot be retrofitted, because retrofitting it recovers nothing.

Pitfall. Disabling the _Default sink is the standard cost optimization once an aggregated sink is in place, and it is correct — but it must be done after the aggregated sink is verified to be receiving entries, not in the same change.

17.7 Log Buckets §

A log bucket is the storage object Cloud Logging writes entries into. It is regional, it has a retention period, and it is not a Cloud Storage bucket despite the name.

Every project starts with two, and neither can be deleted. The _Required bucket "stores log entries that are required for compliance or auditing purposes", retains for 400 days, and "you can't modify which log entries are stored in this log bucket". The _Default bucket "stores log entries that aren't automatically stored in the _Required log bucket" at 30 days, and you can modify what goes into it.

Custom buckets are where your design lives, and four properties are set at creation:

gcloud logging buckets create lb-audit-7y \
  --project=rc-saas-shared-log-01 \
  --location=us-central1 \
  --retention-days=2555 \
  --enable-analytics \
  --description="Organization audit log archive, seven-year retention."

--restricted-fields is the underused one. It marks fields such as jsonPayload or protoPayload as requiring an additional permission (roles/logging.fieldAccessor) to read, so an operator can query metadata without reading payload contents that may contain personal data.

Pitfall. Bucket region is a data residency decision made once, silently, by whoever ran the create command. Set it in Terraform with the estate's residency requirement encoded, because the remedy for a wrong region is a new bucket and a re-pointed sink — the existing entries do not move.

17.8 Log Views §

A log view is a named, filtered window onto a log bucket, with its own IAM policy. It is how you let a team read their logs from a central bucket without reading everyone else's.

Views are the reason centralization does not force over-permissioning. Granting roles/logging.viewer on the logging project grants every log in it. Granting roles/logging.viewAccessor on one view grants exactly the entries the view's filter selects.

gcloud logging views create lv-app-team \
  --bucket=lb-audit-7y \
  --location=us-central1 \
  --project=rc-saas-shared-log-01 \
  --log-filter='resource.labels.project_id="rc-saas-prod-app-01"' \
  --description="Production application project logs, for the app team."

gcloud logging views add-iam-policy-binding lv-app-team \
  --bucket=lb-audit-7y --location=us-central1 \
  --project=rc-saas-shared-log-01 \
  --member="group:gcp-app-developers@rickcollette.domain" \
  --role=roles/logging.viewAccessor

The filter language for a view is deliberately narrow — it supports SOURCE(), resource.type, and LOG_ID() — because a view filter must be cheap to evaluate on every read. Complex analysis belongs in a query, not a view.

This estate's view set is one view per environment plus one per business unit in the enterprise estate, all over the same bucket. Views are cheap; buckets are not, because each one is a separate retention and residency decision.

Pitfall. A view restricts what a principal reads through the Logging API. It does not restrict what they read through a linked BigQuery dataset (§17.18) or an export destination (§17.15), each of which has its own IAM. A view is one of three doors, and securing only it is common and wrong.

17.9 Log Sinks §

A sink is a filter plus a destination, evaluated by the Logs Router on every matching entry. Sinks are the only mechanism by which a log leaves the project it was written in.

Five destination types are supported, and each needs a different grant to the sink's writer identity:

The writer identity is created with the sink and must then be granted. Cloud Logging generates a service account for the sink; a sink whose writer identity has not been granted on the destination is created successfully, reports no error, and routes nothing. This is the most common logging misconfiguration in existence.

gcloud logging sinks create sink-org-audit-archive \
  storage.googleapis.com/rc-saas-shared-log-archive-01 \
  --organization=123456789012 \
  --include-children \
  --log-filter='logName:"logs/cloudaudit.googleapis.com"' \
  --description="All audit logs, organization-wide, to immutable archive."

gcloud logging sinks describe sink-org-audit-archive \
  --organization=123456789012 --format="value(writerIdentity)"

Terraform does the grant for you if you wire it, which is the argument for managing sinks as code:

resource "google_logging_organization_sink" "audit_archive" {
  name             = "sink-org-audit-archive"
  org_id           = "123456789012"
  destination      = "storage.googleapis.com/${google_storage_bucket.log_archive.name}"
  include_children = true
  filter           = "logName:\"logs/cloudaudit.googleapis.com\""
}

resource "google_storage_bucket_iam_member" "sink_writer" {
  bucket = google_storage_bucket.log_archive.name
  role   = "roles/storage.objectCreator"
  member = google_logging_organization_sink.audit_archive.writer_identity
}

Exclusions subtract from a sink's filter. "Log entries that match any of the exclusion filters will not be exported", and the flag is repeatable, which makes --exclusion the correct tool for dropping a single noisy log source without rewriting the inclusion filter.

Pitfall. --intercept-children is not --include-children. Include copies child logs to this sink; intercept additionally prevents them from matching sinks in child resources, "except _Required sinks". Setting intercept on an organization sink silently disables every project team's own export.

17.10 Centralized Logging §

Centralized logging means one aggregated sink at the organization writing every project's logs to storage a single team controls, rather than each project managing its own retention.

The whole design is one flag. An organization-scoped sink with --include-children matches log entries from every folder and project beneath it, including projects created after the sink. That is what makes it a control rather than a convention: a new project cannot opt out.

Three destinations, each answering a different question:

Run all three from separate sinks, not one sink with three copies. Separate sinks let you filter each destination independently — everything to the archive, audit logs only to the analysis bucket, high-signal events only to Pub/Sub — and a failure in one does not affect the others.

The organization policy that backstops this is constraints/gcp.disableCloudLogging, which must not be allowed anywhere, and constraints/gcp.detailedAuditLoggingMode, which extends audit log content where an estate needs full request and response detail.

Pitfall. An aggregated sink at the organization does not capture logs from resources outside the organization — a personal project a developer used, or a project in another organization. Centralization is only as complete as the resource hierarchy (§1.6), and a project outside it is invisible to every control in this chapter.

17.11 Cross-Project Logging §

Cross-project logging is the mechanics of a log written in one project being stored, read, or queried in another.

Three distinct mechanisms exist, and they are frequently confused:

• A sink routing across projects — the normal case, covered in §17.9. The writer identity of the source project's sink is granted on the destination project's resource.
• A custom writer identity — --custom-writer-identity on a sink, permitted only when the destination is a log bucket in a different project. It lets you use a service account you control rather than a generated one, which matters when the destination project's IAM is managed by a different team on a slower change cycle.
• A log scope — gcloud logging scopes, a named set of projects and views that the Logs Explorer queries together, so an analyst can search across an application's projects without an export.

Reading across projects is an IAM question, not a routing one. Once entries are in the central bucket, access is granted per view (§17.8). Nothing needs to be copied a second time.

The permission to create a sink is roles/logging.configWriter on the source scope, and this book grants it at the organization to the platform team only. A project team that can create sinks can create a sink to a destination outside the estate.

Pitfall. Routing logs to another project as a sink destination re-evaluates them against that project's sinks, which can produce loops or duplicate storage if the destination project also has an aggregated sink pointing back. Route to a concrete destination — a bucket, a dataset, a topic — rather than to a project, unless you specifically want re-evaluation.

17.12 Logging Projects §

A logging project is a project whose only purpose is to hold log destinations. §2.22 owns the design argument — why the aggregated sink lives at the organization, and why the destination must be administered by a team that does not administer the workloads. This section covers what goes in it.

The estate's logging project is rc-saas-shared-log-01, and it holds exactly four kinds of resource:

Its region is a residency decision, not a latency one. Log buckets cannot be moved after creation, so the logging project's region is fixed on the day the estate is built. The SaaS estate uses us-central1; the enterprise estate uses us-east4 (its primary region) to keep audit data inside the same jurisdiction as the workloads it describes.

Nothing runs in it. No workload, no build, no service account used by an application. The only principals with any role in the logging project are the platform team (roles/logging.admin), the security team (roles/logging.privateLogViewer), and the sink writer identities.

One project or one per environment? One. Splitting by environment splits the audit trail exactly along the line an investigation needs to cross, and the access control that splitting is meant to provide is what log views (§17.8) already give you at zero structural cost.

Pitfall. The logging project is a single point of failure for evidence, so its own audit logs must be routed somewhere else — in practice, to the Cloud Storage archive with Bucket Lock (§11.16), which is the one copy that a compromise of the logging project cannot rewrite.

17.13 Log Retention §

Retention is how long a log bucket keeps an entry before deleting it, and in Cloud Logging it is a per-bucket setting with two fixed exceptions.

The two fixed buckets bound your floor and your default:

Retention is billed after 30 days. Cloud Logging charges for ingestion, and storage beyond the first 30 days is charged separately per GB-month. That pricing shape is what should drive your design: keep everything for 30 days because it is already paid for, and make a deliberate, filtered decision about what earns a longer bucket.

This book's three-tier retention, expressed as three sinks into three buckets:

• Audit logs — seven years (--retention-days=2555) in lb-audit-7y, because SOC 2 and most contractual obligations land between three and seven years (Chapter 30).
• Security-relevant application logs — one year in lb-ops-1y. Long enough to cover a dwell time that goes undetected for months.
• Everything else — 30 days in _Default, which costs nothing extra.

Judgment. Buy retention with the filter, not with the retention period. A seven-year bucket holding every debug line is a five-figure annual line item; a seven-year bucket holding logName:"logs/cloudaudit.googleapis.com" for a mid-sized estate is not.

Pitfall. "You also can't extend the retention period of a log bucket that is in a folder or organization." Folder- and organization-level buckets do not behave like project-level ones, which is a good reason for the central buckets in §17.12 to live in a project.

17.14 Immutable Logging §

Immutability in Cloud Logging means an entry cannot be deleted, shortened, or altered before its retention period expires — including by the person who configured it.

Locking a log bucket is the mechanism, and it is permanent. Google states it twice, and both statements matter: "Locking a log bucket is irreversible", and "After a retention policy is locked, you can't delete the bucket until every log entry in the bucket has fulfilled the bucket's retention period."

gcloud logging buckets update lb-audit-7y \
  --project=rc-saas-shared-log-01 \
  --location=us-central1 \
  --locked

What locking does and does not do is the distinction teams get wrong:

A locked bucket with a deletable sink is not an immutable audit trail. The sink is the weak link, which is why §17.20 treats sink modification as the control-plane event to alert on, and why the Cloud Storage archive — protected by Bucket Lock (§11.16), a genuinely separate service with separate IAM — is the second copy.

CMEK adds cryptographic control but not immutability. --cmek-kms-key-name on a bucket, or gcloud logging settings update --kms-key-name at the organization, encrypts log storage with a key you hold. Once set, "the CMEK settings cannot be disabled later by updating the log bucket. Changing the KMS key is allowed." Grant the Log Router's service account roles/cloudkms.cryptoKeyEncrypterDecrypter on the specific key, per §14.8.

Pitfall. Lock in production only after retention is final. A locked seven-year bucket created with a typo in the retention days is a seven-year mistake, and the only remedy is a second bucket and a re-pointed sink.

17.15 BigQuery Log Exports §

A BigQuery sink writes log entries into a dataset as rows, where they can be queried with SQL and joined against anything else in the warehouse.

This is an export, and you pay twice. Entries are ingested into Cloud Logging and then written again into BigQuery, incurring BigQuery storage and query cost on top of Logging's. That is the essential difference from analytics on a log bucket (§17.18), where "you don't ingest your log data into BigQuery" and there are no additional ingestion or storage charges.

Use partitioned tables, always. Without --use-partitioned-tables the sink writes date-sharded tables, and every query scans more than it needs:

gcloud logging sinks create sink-org-audit-bq \
  bigquery.googleapis.com/projects/rc-saas-shared-log-01/datasets/audit_logs \
  --organization=123456789012 \
  --include-children \
  --use-partitioned-tables \
  --log-filter='logName:"logs/cloudaudit.googleapis.com%2Factivity"'

Choose BigQuery export over analytics only for a specific reason: joining logs to non-log data (a customer table, an asset inventory, an HR feed), retaining beyond a log bucket's 3650-day ceiling, or feeding an existing warehouse pipeline. For querying logs as logs, §17.18 is cheaper and simpler.

The dataset's own access control is separate. roles/bigquery.dataViewer on the dataset reads every exported entry regardless of any log view. Set a dataset-level default table expiration matching your retention policy, because BigQuery does not inherit the log bucket's.

Pitfall. Log entry schemas evolve. BigQuery sinks add columns as new fields appear, and a field whose type changes produces a second column with a suffix. Queries written against a column name from last year can silently return nothing rather than failing.

17.16 Pub/Sub Log Exports §

A Pub/Sub sink publishes each matching log entry as a message, which is how logs reach anything that is not a Google Cloud storage service — a SIEM, a webhook, a stream processor.

Pub/Sub is transport, not storage. Messages are retained only until acknowledged, up to a configurable maximum (seven days by default). A consumer that is down long enough loses entries permanently, and there is no replay.

gcloud logging sinks create sink-org-security-pubsub \
  pubsub.googleapis.com/projects/rc-saas-shared-log-01/topics/logs-siem \
  --organization=123456789012 \
  --include-children \
  --log-filter='logName:"logs/cloudaudit.googleapis.com" AND severity>=NOTICE'

Filter hard at the sink. Every message is billed and every message is delivered, so an unfiltered organization-wide Pub/Sub sink is simultaneously the most expensive and the least useful configuration available. Route what an analyst would investigate, not what a system emits.

Always pair it with a durable sink. The Pub/Sub stream is the real-time path; the log bucket or Cloud Storage archive is the record. If the two disagree — and after a consumer outage they will — the durable copy is the evidence.

The writer identity needs roles/pubsub.publisher on the topic, and the consumer needs roles/pubsub.subscriber on the subscription and nothing else. A consumer with topic-level permissions can create subscriptions and change the delivery topology.

Pitfall. Pub/Sub delivery is at-least-once. A SIEM that counts events without deduplicating on insertId will over-report, and an automated response keyed on message arrival will act more than once (§16.16).

17.17 SIEM Integration §

A SIEM integration is the pipeline that carries Google Cloud logs into an analysis platform, and it is the consumer half of the export mechanisms in §17.16 and §16.15. Security Command Center's own export configuration is §16.15's subject; this section is what happens after the message is published.

Two first-party paths and one universal one:

Google's Splunk path is a documented, supported template, not a script you write: the Pub/Sub to Splunk Dataflow template reads a subscription and writes to Splunk's HTTP Event Collector, and the "Stream logs from Google Cloud to Splunk" reference architecture covers the production shape, including a dead-letter topic for events the collector rejects. Use it rather than a custom consumer; the failure handling is the hard part and it is already written.

Authenticate the connector with Workload Identity Federation. A SIEM running outside Google Cloud federates its own identity into a service account holding roles/pubsub.subscriber on one subscription (§4.13). A downloaded service account key for a log connector is a long-lived credential that reads your entire audit trail, and it is the single most common exception teams make to the no-keys rule.

Correlate findings with logs, not instead of them. The SCC finding stream (§16.15) says a detector fired; the audit log says what the principal actually did before and after. A SIEM that ingests only one of the two answers half of every question.

Pitfall. Ingestion volume is the SIEM's cost driver and the sink's filter is the only place to control it. Teams route everything and then filter at the SIEM, having already paid for ingestion twice — once in Pub/Sub and once in the SIEM's licensing.

17.18 Security Analytics §

Security analytics here means running SQL over your own logs. Google's product for it is the analytics capability on a log bucket, presented on current documentation as Observability Analytics and still exposed by the older name Log Analytics in some surfaces and by the --enable-analytics flag.

What it is: "Using Observability Analytics, you can generate insights by running queries that group and aggregate your log data." A bucket opted in gets a SQL query surface, and optionally a linked BigQuery dataset.

The linked dataset is the part that changes the economics. "When you create a linked BigQuery dataset for a log bucket, you don't ingest your log data into BigQuery. Instead, you get read access to the log data stored in your log bucket through the linked BigQuery dataset." And: "There are no BigQuery ingestion or storage costs when you upgrade a bucket to use Observability Analytics and then create a linked BigQuery dataset."

gcloud logging links create audit_linked \
  --bucket=lb-audit-7y \
  --location=us-central1 \
  --project=rc-saas-shared-log-01 \
  --description="Linked dataset for SQL over the seven-year audit bucket."

Upgrading an existing bucket has three conditions: the bucket "was created at the Google Cloud project level", it "is unlocked unless it is the _Required bucket", and "There aren't pending updates to the bucket". The second condition is the trap — a bucket you locked for immutability (§17.14) cannot be upgraded afterward.

Enable analytics at bucket creation. It cannot be disabled once enabled, and it cannot be enabled after locking, so the ordering is: create with --enable-analytics, verify, then lock.

Pitfall. The linked dataset grants read access through BigQuery IAM, which is a completely separate door from the log views in §17.8. A principal with roles/bigquery.dataViewer on the linked dataset reads every entry in the bucket, view filters notwithstanding.

17.19 Audit Trail Design §

An audit trail is a design, not a product: a set of decisions about what is recorded, where it goes, how long it lives, and what makes it trustworthy.

Five questions decide the whole thing, and answering them in this order avoids most rework:

1. What must be answerable, and how far back? Regulatory retention (Chapter 30) sets the floor; incident dwell time sets the practical minimum, and it is usually longer.
2. Which Data Access logs are worth their cost? §17.3. This is the only expensive answer, and it is per service.
3. Where does the trail live, and who administers it? §17.12 and §2.22.
4. What makes it tamper-evident? §17.14, plus the second copy in a different service.
5. Who can read it, at what granularity? §17.8.

Completeness is a coverage question, not a filter question. An audit trail that omits a project, a folder, or an organization is not partially complete — it has a blind spot exactly where an attacker who understands the estate would work. Verify coverage by enumerating projects and confirming each is under the aggregated sink's scope, not by reading the sink's filter.

Correlate on identity. protoPayload.authenticationInfo.principalEmail is the join key across every service's audit log. An estate that binds roles to groups (§3.4) and uses service accounts with descriptive names gets a readable trail for free; one that uses shared accounts gets an unreadable one no tooling can fix.

Pitfall. Impersonation breaks naive attribution. When a human impersonates a service account (§4.7), the audit entry's principalEmail is the service account, and the human appears only in authenticationInfo.serviceAccountDelegationInfo. A query that reports only principalEmail attributes every impersonated action to a robot.

17.20 Protecting Logs from Administrators §

The threat this section addresses is a privileged insider — or an attacker holding a privileged credential — who tries to remove the record of what they did. Retention (§17.13) and locking (§17.14) supply the mechanisms; this is the design and the honest limits.

Five moves erase evidence, in roughly increasing order of subtlety:

Separation of duties is the actual control. Nobody who administers a workload project may hold roles/logging.configWriter at the organization or roles/logging.admin in the logging project. The platform team owns routing; the security team reads; the workload teams have neither. Enforce it with a deny policy (§3.17) rather than by omission, because a role grant is one SetIamPolicy away.

Alert on the control plane, from a log that cannot be turned off. Every event above is an Admin Activity entry (§17.2), which by construction survives the attempt to suppress it. §18.15 owns the alert policy; this section owns the list of what to alert on.

The honest limit: an organization administrator can still win. A principal with organization-level administrative rights can grant themselves anything, and the aggregated sink they control is within their reach. The only structural answer is a copy outside the blast radius — the Cloud Storage archive with Bucket Lock in a separate project with separate administrators, or an export into a system in a different trust domain entirely (§17.17). Say this out loud in your threat model rather than claiming the in-organization design is tamper-proof.

Pitfall. Teams protect the log bucket and forget the archive's IAM. A Cloud Storage archive whose bucket is retention-locked but whose project can be deleted by the same administrator has bought nothing; §11.16 and the project deletion protection in §2.9 are both required.

17.21 Log Cost Management §

Cloud Logging bills for ingestion by volume and for storage beyond the first 30 days. Both are controllable, and the controls are the same filters you already write.

Four levers, in the order they should be applied:

1. Exclude at the _Default sink. High-volume, low-value entries — load balancer health check requests, verbose framework logging, expected Policy Denied noise in a single project — never need to be stored. Excluding at the sink means they are never ingested into the bucket.
2. Do not enable DATA_READ on allServices. §17.3. This one decision moves the bill more than the other three combined.
3. Right-size retention by filter. §17.13: a small bucket held for years costs less than a large one held for months.
4. Prefer analytics over BigQuery export. §17.18: the linked dataset has no additional ingestion or storage cost; the export has both.

Admin Activity, System Event, and the _Required bucket carry no ingestion charge, so the compliance floor of this chapter's design is free. Everything you pay for is a deliberate addition, which makes cost review a review of those decisions rather than a hunt.

Measure before cutting. Console → Logging → Log storage reports ingestion by bucket, and a log-based metric grouped by resource.type (§18.2) finds the specific source of a spike. Cutting by intuition removes the low-volume logs that were cheap and valuable.

Judgment. Never reduce audit log coverage to reduce cost. If the audit trail is genuinely unaffordable, the problem is the retention and the destination count, not the coverage — cut the archive tier, cut the BigQuery export, cut application log verbosity, and leave cloudaudit.googleapis.com alone.

Pitfall. An exclusion filter written to suppress a noisy application log is easy to write too broadly — severity<WARNING on the _Default sink drops INFO-level audit entries as well, and most audit entries are INFO. Scope every exclusion with a logName or resource.type clause.

Chapter Summary §

• Cloud Audit Logs come in four types; Admin Activity and System Event are always on and free, Data Access is off by default, and Policy Denied cannot be disabled but can be excluded.
• Data Access logging is configured in the IAM policy's auditConfigs, per service and per log type, and DATA_READ on allServices is the largest logging bill most estates generate.
• Everything routes through the Logs Router, and a sink cannot route entries received before it existed — so the aggregated sink is created first, before any workload.
• The _Required bucket keeps a subset of audit logs for 400 days and cannot be changed; the _Default bucket keeps everything else for 30 days and can.
• Log buckets are regional and their region is fixed at creation, making it a residency decision made once.
• Log views give per-audience read access over a shared bucket, but they are only one of three doors — export destinations and linked datasets each have their own IAM.
• A sink's writer identity must be granted on the destination; without the grant the sink is created, reports no error, and routes nothing.
• --include-children copies child logs; --intercept-children additionally stops them matching child sinks, which silently disables project teams' own exports.
• Locking a log bucket is irreversible and prevents shortening retention or deleting the bucket — but not deleting the sink that fills it.
• CMEK on a log bucket cannot be removed once set, only rotated to a different key.
• BigQuery export pays for ingestion and storage twice; a linked dataset over an analytics-enabled bucket has neither cost.
• An analytics-enabled bucket must be upgraded before it is locked, because a locked bucket cannot be upgraded.
• Pub/Sub is transport, never the record: it is at-least-once, retention is bounded, and there is no replay.
• Impersonation puts the service account in principalEmail and the human in serviceAccountDelegationInfo; a query reading only the former misattributes every impersonated action.
• An organization administrator can still defeat an in-organization audit trail; the structural answer is a locked copy in a separately administered project or trust domain.

Security Checklist §

Sources §

• https://cloud.google.com/logging/docs — Cloud Logging documentation hub (last validated 2026-09-03)
• https://cloud.google.com/logging/docs/audit/understand-audit-logs — the four audit log types and their default state (last validated 2026-09-03)
• https://cloud.google.com/logging/docs/audit/configure-data-access — the auditConfigs structure and exempted principals (last validated 2026-09-03)
• https://cloud.google.com/logging/docs/routing/overview — Logs Router behavior, _Required and _Default sinks (last validated 2026-09-03)
• https://cloud.google.com/logging/docs/buckets — bucket retention range, locking, and region immutability (last validated 2026-09-03)
• https://cloud.google.com/logging/docs/store-log-entries — _Required 400-day and _Default 30-day retention (last validated 2026-09-03)
• https://cloud.google.com/logging/docs/export/configure_export_v2 — sink writer identity and per-destination roles (last validated 2026-09-03)
• https://cloud.google.com/logging/docs/log-analytics — Observability Analytics and linked BigQuery datasets (last validated 2026-09-03)
• https://cloud.google.com/dataflow/docs/guides/templates/provided/pubsub-to-splunk — the Pub/Sub to Splunk Dataflow template (last validated 2026-09-03)
• https://cloud.google.com/architecture/stream-logs-from-google-cloud-to-splunk — production reference architecture for the Splunk path (last validated 2026-09-03)
• https://cloud.google.com/chronicle/docs/ingestion/cloud/ingest-gcp-logs — Google Security Operations ingestion of Google Cloud logs (last validated 2026-09-03)