Chapter 9

Google Kubernetes Engine

Scope. This chapter covers Kubernetes on Google Cloud end to end: cluster architecture and modes, networking, identity, workload and node security, supply chain enforcement, operations, and multi-cluster management. It defers the VM primitive underneath every node to Chapter 8, load balancers and Cloud Armor to Chapter 7, Secret Manager's own resource model to Chapter 13, Cloud KMS mechanics to Chapter 14, and attestation authoring to Chapter 37 — each section cites the owning chapter rather than repeating it. Prerequisites. Chapter 8 (Compute Engine primitives), §4.12 (Kubernetes Workload Identity), §5.4 (Subnets), §2.30–§2.31 (organization policy). Verified against. Google Cloud console and API surface as of 2026-09, Cloud SDK 583.0.0, hashicorp/google provider 8.x; see sources at end.

Kubernetes solves orchestration; it does not solve isolation, identity, or supply chain integrity, and a cluster that runs correctly is not the same thing as a cluster that resists a hostile tenant, a compromised image, or a stolen node credential. GKE narrows the gap between those two states by managing the control plane, wiring node identity through Compute Engine's attached-service-account model, and layering GCP-native controls — Workload Identity Federation, Binary Authorization, VPC-native networking, Shielded Nodes — on top of upstream Kubernetes primitives that were never designed with a multi-tenant, internet-facing default in mind.

The chapter follows a cluster's own build order: architecture and mode choice, networking and identity, workload and node security, ingress and supply chain, operations, then multi-cluster and fleet management. Two sections deliberately show less than a first read expects: §9.11 covers only the cluster-configuration side of Workload Identity Federation for GKE because §4.12 already owns the identity mechanics, and §9.7 covers only GKE's consumption of VPC-native networking because §5.4 already owns secondary ranges. Both patterns repeat throughout the book — a capability is described once, where it is general, and consumed by reference everywhere else — and GKE is the chapter where the reader will meet it most often, because a GKE cluster touches nearly every other control in this book at once: IAM, networking, KMS, Secret Manager, and organization policy all converge on a single resource.

The running example is the SaaS estate's production cluster, rc-saas-prod-gke-01 in rc-saas-prod-app-01, built as a private, VPC-native, Workload-Identity-enabled Standard cluster on the Regular release channel, with an Autopilot alternative shown wherever the operational trade-off matters.

9.1 Kubernetes on GCP §

Google originated Kubernetes from its internal Borg system and remains the largest single contributor to the upstream project, which shapes GKE's default posture: GKE tracks upstream releases closely and exposes GA Kubernetes APIs without a translation layer. What GKE adds is the control plane you do not operate, a managed node lifecycle (autoscaling, autorepair, autoupgrade), and a set of Google Cloud–native integrations — IAM, VPC, Cloud KMS, Artifact Registry, Cloud Logging and Monitoring — that a self-managed cluster on Compute Engine would have to wire by hand.

Security posture. The default posture has moved sharply toward secure defaults over successive GKE versions: new Standard clusters enable Shielded Nodes, and Autopilot enables Workload Identity Federation, VPC-native networking, and NetworkPolicy by default. The residual risk is almost entirely in what an operator disables to move faster, not in what GKE ships open.

Pitfall. Treating GKE as "just Kubernetes" and importing upstream tutorials wholesale skips the GCP-specific controls — Workload Identity, Shielded Nodes, VPC-native ranges — that the rest of this chapter exists to apply.

9.2 GKE Architecture §

A GKE cluster has two halves with different ownership. The control plane — API server, scheduler, controller manager, etcd — runs in a Google-managed tenant project, is billed by cluster-management fee (with a free tier), and is patched and backed up by Google; you never SSH to it. The nodes are Compute Engine instances in your own project, in a Google-managed managed instance group per node pool, and you are responsible for their OS image, service account, and (for Standard clusters) upgrade cadence.

Zonal vs. regional. A zonal cluster runs one control plane replica in one zone; a regional cluster (the default for new clusters, and the only mode Autopilot uses) runs replicated control plane instances across three zones in the region, surviving a single zone outage. Node pools in a regional cluster are still zonal in placement unless created as regional (spread across the region's zones); prefer regional node pools for anything that must tolerate a zone failure.

GKE Sandbox adds a second isolation layer inside a node: gVisor re-implements the Linux kernel's system call surface in userspace, so a sandboxed container never issues a syscall directly to the host kernel. Standard clusters enable it per node pool with --sandbox=type=gvisor, at which point every pod scheduled to that pool is sandboxed; Autopilot allows requesting it per pod. It cannot run privileged containers, host-path volumes, VolumeDevices, port-forwarding, or Seccomp/AppArmor/SELinux profiles, and only CUDA-based GPU workloads on specific accelerator models are supported — reserve it for multi-tenant clusters running workloads you did not author.

Control plane keys and CA material. gcloud container clusters create also accepts --cluster-ca, --etcd-api-ca, and --etcd-peer-ca to back the cluster's certificate authorities with a Certificate Authority Service pool instead of GKE's own internal CA, and --control-plane-disk-encryption-key to encrypt control plane disks with a Cloud KMS key you hold (§14.8) — both are advanced options for estates with a CA or key-custody requirement beyond GKE's defaults, not a default configuration.

IAM boundary. roles/container.admin grants full cluster lifecycle control including RBAC bootstrap; roles/container.developer grants workload operations without cluster administration; roles/container.viewer is read-only. Grant roles/container.admin at the project, never the organization, and prefer roles/container.clusterViewer for anyone who only needs kubectl get.

9.3 Standard vs. Autopilot §

Autopilot is GKE's mode where Google manages node provisioning, sizing, and most of the node-level security surface; Standard gives you the node pool primitives directly. The choice is not "more secure vs. less secure" — both run the same control plane — it is how much operational and security configuration remains in your hands.

What Autopilot removes from your responsibility, and therefore from your attack surface: node OS choice and patching (Container-Optimized OS only), node service account attachment (a minimal one is enforced and cannot be changed after creation), Shielded Nodes and VPC-native networking (both mandatory, not optional), and privileged pod admission (blocked by default; --enable-autopilot-privileged-admission is a documented escape hatch you should treat as an incident-worthy exception). Autopilot bills per pod resource request rather than per node, which also removes the incentive to over-pack nodes to save cost — a common source of noisy-neighbor privilege escalation in Standard clusters.

What Autopilot still leaves to you: Kubernetes RBAC, namespace design, network policy, Workload Identity bindings, Secret management, Binary Authorization policy, and everything at the application layer. Autopilot is not a compliance shortcut; it removes infrastructure-layer decisions, not workload-layer ones.

When Standard is the right choice: GPU/TPU topologies Autopilot does not yet support, DaemonSets that need host-level access for security tooling, sole-tenant nodes, custom node images, or cost models that depend on sustained-use committed node pools rather than per-pod billing.

Cloud Run (Chapter 10) is the right choice instead of either when the workload is a stateless container needing no Kubernetes API surface at all.

gcloud container clusters create-auto rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --release-channel=regular

Pitfall. create-auto accepts far fewer flags than create — most security posture is fixed, not configured — so review the Autopilot defaults against your requirements before assuming a flag you rely on in Standard is available.

9.4 Control Plane Architecture §

The control plane's IP exposure and its authentication model are two independent decisions, both fixed at creation for the exposure choice and mutable for authorized networks.

Endpoint model (current, GA). GKE now offers a DNS-based endpoint and an IP-based endpoint, and DNS-based is Google's current recommendation: it resolves to a stable, immutable FQDN scoped to the cluster, integrates with IAM and VPC Service Controls, and needs no bastion host to reach from outside the VPC. --enable-dns-access turns it on; combine it with --enable-authorized-networks-on-private-endpoint to enforce authorized networks against DNS-based access, since without it the DNS endpoint is reachable from IAM-authorized callers regardless of network path. The IP-based endpoint (--enable-ip-access) keeps the older model: an internal IP always, and a public IP unless --enable-private-endpoint is also set. --enable-google-cloud-access widens the public IP-based endpoint to any Google Cloud–owned public address, which is rarely what you want in production.

gcloud container clusters create rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --enable-dns-access \
  --enable-authorized-networks-on-private-endpoint \
  --enable-master-authorized-networks \
  --master-authorized-networks=10.128.0.0/20

API surface. The API server exposes standard Kubernetes RBAC-gated endpoints; GKE additionally fronts it with IAM for gcloud/kubeconfig token minting (container.clusters.get and the credential helper), so a caller needs both an IAM grant to obtain credentials and a Kubernetes RBAC grant to act once connected (§9.13).

Pitfall. Leaving both --no-enable-master-authorized-networks and the IP-based public endpoint enabled makes the control plane reachable from 0.0.0.0/0, authenticated only by whatever credential the caller presents — authorized networks is the network-layer control that authentication alone does not replace.

9.5 Node Pools §

A node pool is a set of nodes sharing one machine configuration, managed as a Google-owned managed instance group. A cluster needs at least one node pool and may run many, each with its own machine type, disk, service account, taints, and autoscaling range — the mechanism for placing distinct workload classes (general-purpose, GPU, spot) without separate clusters.

gcloud container node-pools create np-default \
  --cluster=rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --machine-type=n2-standard-4 \
  --num-nodes=2 \
  --enable-autorepair \
  --enable-autoupgrade

This book names node pools np-<purpose> (np-default, np-system, np-gpu, np-spot), matching each pool's role rather than its machine type, so the name stays accurate across a resize.

Pitfall. Deleting the last node pool in a cluster does not delete the cluster, but leaves a control plane with nowhere to schedule anything — budget node pool changes as a rolling operation, never a delete-then-create.

9.6 Cluster Networking §

GKE networking has three address families to plan before creation: node IPs (regular VPC subnet addresses, one per node), pod IPs (from the pod secondary range, many per node), and Service IPs (from the services secondary range, virtual and never routed). Traffic between pods on different nodes crosses the VPC using those alias IP ranges directly, with no overlay network required — the property that makes GKE's networking visible to VPC firewall policy and flow logs in the first place.

Service types and their exposure: ClusterIP (internal only, the default and the correct default), NodePort (opens a port on every node's VPC-routable IP — avoid it outside debugging), and LoadBalancer (provisions a GCP load balancer; see §9.23). Ingress and the Gateway API (§9.21, §9.22) sit above Services for HTTP routing.

DNS. GKE runs kube-dns or, on newer clusters, Cloud DNS for GKE as the in-cluster resolver; --cluster-dns=CLOUD_DNS moves resolution off the in-cluster pods entirely, which removes a pod-level attack surface and a single point of failure at the cost of a dependency on the Cloud DNS API being reachable from every node.

Pitfall. NodePort Services are reachable from anywhere the node's VPC firewall policy allows, which is usually far broader than the pod's intended audience — a difference reviewers miss because the Service manifest itself carries no IP restriction.

9.7 VPC-Native Clusters §

VPC-native (alias IP) is now the only networking mode gcloud container clusters create offers for new clusters; the legacy routes-based mode is retired. §5.4 owns secondary ranges and the four-unusable-address rule; this section covers only how GKE consumes them.

gcloud container clusters create rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --network=vpc-prod-global \
  --subnetwork=snet-prod-usc1-app \
  --enable-ip-alias \
  --cluster-secondary-range-name=pods \
  --services-secondary-range-name=services \
  --default-max-pods-per-node=64

This book's fixed CIDR plan (§5.6): nodes 10.140.0.0/20, pods 10.144.0.0/14, services 10.148.0.0/20.

The pod range bounds cluster capacity, not the node range. GKE allocates one /24 (256 addresses) per node from the pod secondary range by default; --default-max-pods-per-node shrinks that per-node allocation (to as low as 8) in exchange for supporting more nodes from the same range. A /14 pod range holds roughly 262,144 addresses; at the default /24 per node that is 1,024 nodes, and at --default-max-pods-per-node=64 (which still rounds up to the next power-of-two-aligned block internally) considerably more. Size --default-max-pods-per-node down deliberately when the node range, not the pod range, is the actual ceiling you are planning against.

Exhaustion at scale-out looks like new nodes failing to register with no application-visible symptom until you read the node or cluster autoscaler events — the pod range has run out of per-node blocks to hand out. There is no in-place fix: a secondary range can be expanded via --additional-pod-ipv4-ranges on cluster update, but a secondary range already in use by a cluster cannot be removed (§5.4), so oversizing pods at plan time is cheaper than correcting it later.

9.8 Private Clusters §

A private cluster restricts node IPs to internal addresses; it does not by itself restrict the control plane, which is the separate decision in §9.4 and §9.10. Google Cloud's private-cluster documentation frames the intent directly: nodes are provisioned only with internal IP addresses so that no external client can reach a node's kubelet or workload ports directly.

gcloud container clusters create rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --enable-private-nodes \
  --master-ipv4-cidr=10.140.16.0/28 \
  --enable-dns-access \
  --enable-master-authorized-networks \
  --master-authorized-networks=10.128.0.0/20

The /28 supplied to --master-ipv4-cidr is the reference plan's control plane range for this book: 10.140.16.0/28, distinct from the node range 10.140.0.0/20 (§5.6) and stated once here for reuse anywhere this cluster is referenced.

Security posture. Private nodes still reach the internet only through Cloud NAT or a proxy you provision (§5.20) — there is no implicit egress path, unlike a public node with an external IP. Nodes still reach Google APIs through Private Google Access (§5.17), which a private cluster requires on its subnet to pull images and reach the control plane's internal path.

9.9 Private Nodes §

Private nodes is the setting inside a private cluster (--enable-private-nodes) that removes external IPs from the node VMs themselves. It is independent of the control-plane endpoint decision (§9.4): a cluster can be VPC-native without being private, and its control plane can be public while its nodes are not.

Every new cluster should default to private nodes, whatever the control plane does. A node with an external IP is directly reachable from the internet subject only to firewall rules, and it is the node — not the control plane — that runs your workloads and holds the node service account. constraints/container.managed.enablePrivateNodes (§2.30) enforces this org-wide so it is not a per-cluster choice.

Private nodes need an egress path provisioned first, because they still must reach the internet and Google APIs:

• Cloud NAT (§5.20) for anything outside Google — pulling a public base image, calling a third-party API.
• Private Google Access (§5.17) on the node subnet, for Google APIs including Artifact Registry and Cloud Logging.

Pitfall. Without those two in place, nodes come up unable to pull images or report status, and the cluster sits in PROVISIONING with an error that points at the node pool rather than at the missing NAT gateway. Provision the egress path before the first node pool, not after the cluster is stuck — the failure mode reads as a GKE problem and is a networking one.

9.10 Authorized Networks §

Authorized networks is a CIDR allowlist evaluated at the control plane, in addition to whatever identity and RBAC checks a request passes — it answers "from where," not "as whom." --enable-master-authorized-networks combined with --master-authorized-networks accepts up to 100 CIDR blocks for a private cluster (50 for a public one); leaving it disabled on a public cluster admits the entire internet to attempt authentication.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --enable-master-authorized-networks \
  --master-authorized-networks=10.128.0.0/20,10.129.0.0/20

With a DNS-based endpoint, authorized networks only takes effect once --enable-authorized-networks-on-private-endpoint is also set (§9.4) — the two flags are frequently configured independently and the omission is easy to miss in review.

Pitfall. A CI/CD runner with a rotating IP that is not in the authorized list fails silently from the runner's perspective (connection refused, not an authentication error) — route CI/CD access through a static egress IP or a bastion inside an authorized range rather than adding broad ranges.

9.11 Workload Identity Federation for GKE §

§4.12 owns the identity mechanics of Workload Identity Federation for GKE — the fixed pool, the two binding models, the metadata server and STS exchange, the hostNetwork escalation path, and the connection limit. This section covers only the cluster and node-pool configuration that turns it on.

Cluster and node-pool flags. --workload-pool=PROJECT_ID.svc.id.goog at cluster creation enables the pool; --workload-metadata=GKE_METADATA on every node pool routes metadata-server traffic through the federation path instead of the node's own identity.

gcloud container node-pools create np-default \
  --cluster=rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --workload-metadata=GKE_METADATA

Verifying a pool is active:

gcloud container clusters describe rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --format="value(workloadIdentityConfig.workloadPool)"

What breaks on a legacy node pool. A node pool left without --workload-metadata=GKE_METADATA still serves the raw Compute Engine metadata server to every pod scheduled on it — every workload there effectively runs as the node service account, defeating the pool at the cluster level for that one pool. This is a per-node-pool setting, not inherited from the cluster.

Migrating an existing cluster is two update commands: enable the pool on the cluster, then update every node pool's metadata mode; nodes must be recreated (a rolling node pool upgrade) to pick up the new mode, since it is read at kubelet start.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --workload-pool=rc-saas-prod-app-01.svc.id.goog

gcloud container node-pools update np-default \
  --cluster=rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --workload-metadata=GKE_METADATA

constraints/container.managed.enableWorkloadIdentityFederation (§2.30) enforces the pool org-wide, rejecting cluster creation without it.

resource "google_container_cluster" "prod" {
  name     = "rc-saas-prod-gke-01"
  project  = "rc-saas-prod-app-01"
  location = "us-central1"

  workload_identity_config {
    workload_pool = "rc-saas-prod-app-01.svc.id.goog"
  }
}

9.12 Kubernetes RBAC §

Kubernetes RBAC is a second, independent authorization layer sitting inside the cluster, evaluated after IAM has already decided whether a caller may reach the API server at all (§9.13). A Role or ClusterRole grants verbs on resource types; a RoleBinding or ClusterRoleBinding attaches it to a subject — a user, group, or Kubernetes ServiceAccount.

Security posture. The default posture is permissive by upstream design: a cluster with no RBAC objects beyond the built-ins still ships cluster-admin-equivalent bootstrap bindings for the control plane's own components. constraints/container.managed.disableRBACSystemBindings restricts modification of those system bindings, and constraints/container.managed.disableABAC removes the legacy attribute-based authorizer that predates RBAC and, where still enabled, bypasses it entirely.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: billing
  name: pod-reader
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]

Pitfall. ClusterRoleBinding grants apply cluster-wide regardless of namespace; a binding written for one team's convenience during a Role binding outage frequently outlives the outage and becomes a standing over-grant no one reviews.

9.13 GCP IAM and Kubernetes RBAC §

Two authorization systems gate the same API server, and neither substitutes for the other. IAM decides whether a principal can obtain cluster credentials at all (container.clusters.get, minted via gcloud container clusters get-credentials) and whether they hold coarse project-level roles (roles/container.admin, roles/container.developer, roles/container.viewer); Kubernetes RBAC decides what that authenticated identity can do once inside, at namespace or resource granularity IAM has no concept of.

Google Groups for RBAC lets a ClusterRoleBinding reference a Google Group directly (group:gcp-app-developers@rickcollette.domain), so namespace access follows group membership instead of a Kubernetes-native identity provider — constraints/container.managed.enableGoogleGroupsRBAC gates this org-wide, and it is the right mechanism for human access, leaving Workload Identity Federation (§9.11) for workload access.

gcloud container clusters get-credentials rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1

Least privilege in practice: grant roles/container.developer (not roles/container.admin) to application teams at the project level, and let Kubernetes RBAC narrow further per namespace — a developer who can deploy to billing should not automatically read Secrets in payments.

Pitfall. roles/container.developer alone grants no RBAC inside the cluster; teams new to GKE often stop there and then file a ticket when kubectl returns Forbidden — the IAM grant and an RBAC binding are both required.

9.14 Pod Security §

Pod-level security controls what a container may do to its host: run as root, mount the host filesystem, hold Linux capabilities, or use host namespaces. PodSecurityPolicy, the original Kubernetes admission mechanism for this, was deprecated and removed entirely from upstream Kubernetes in version 1.25; GKE clusters had to disable it before upgrading past that version. Pod Security Admission, a built-in admission controller enforcing the Pod Security Standards, is the replacement, and needs no separate cluster-level enablement flag.

The three standards, applied per namespace via labels, from least to most restrictive: privileged (no restrictions), baseline (blocks known privilege escalations while allowing default pod behavior), and restricted (enforces current pod hardening best practice: non-root, dropped capabilities, no host namespaces, seccompProfile required).

apiVersion: v1
kind: Namespace
metadata:
  name: billing
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

Security posture. Set restricted as the default for every namespace that does not have a documented reason to run privileged workloads, and use audit/warn modes during migration to surface violations before enforcing them.

Pitfall. A namespace with no Pod Security Admission labels runs at the cluster's default level, which is privileged unless a cluster-wide default has been configured — silence is not safety here.

9.15 Network Policies §

A NetworkPolicy is a namespace-scoped allowlist for pod-to-pod traffic; without one, every pod in a cluster can reach every other pod on any port, regardless of namespace. GKE enforces NetworkPolicy objects only when network policy enforcement is turned on — either explicitly (--enable-network-policy, the Calico-based implementation) or implicitly because Dataplane V2 is enabled, in which case Kubernetes NetworkPolicy is always on and needs no separate flag.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  namespace: billing
  name: default-deny-ingress
spec:
  podSelector: {}
  policyTypes: ["Ingress"]

A default-deny policy per namespace, followed by explicit allow rules for required paths, is the correct starting posture — the same allowlist discipline this book applies to VPC firewall policy (§5.14). constraints/container.managed.enableNetworkPolicy (§2.30) requires enforcement to be on org-wide.

Pitfall. NetworkPolicy governs pod IP traffic only; it has no effect on traffic that reaches a pod through a NodePort or hostNetwork path, which bypasses the pod network entirely.

9.16 Dataplane V2 §

Dataplane V2 replaces GKE's traditional iptables-based networking with an eBPF dataplane built on Cilium, giving the kernel itself the packet-routing and policy-enforcement logic instead of a long chain of iptables rules that scales poorly with cluster size. It is enabled by default for new Autopilot clusters and is a flag on Standard.

gcloud container clusters create rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --enable-dataplane-v2 \
  --enable-dataplane-v2-flow-observability

Security posture. With Dataplane V2 enabled, Kubernetes NetworkPolicy is always on — there is no separate enablement step and no way to accidentally leave it off, which removes a whole class of misconfiguration common with the legacy Calico path. Flow observability (--enable-dataplane-v2-flow-observability) surfaces per-connection network telemetry in Cloud Logging, giving pod-to-pod visibility comparable to VPC Flow Logs at the subnet level (§5.4).

Pitfall. Dataplane V2 cannot be disabled after cluster creation, and enabling it on an existing cluster requires recreating every node — plan it at cluster creation, not as a later toggle.

9.17 Secret Management §

Chapter 13 owns Secret Manager as the book's secret store; this section covers only the GKE-specific surface for getting a secret to a pod.

Kubernetes Secrets are not encrypted at rest by default and are not access controlled beyond RBAC. A Kubernetes Secret is base64-encoded, not encrypted — base64 is an encoding, not a cipher — and anyone holding get on secrets in a namespace, through any RBAC binding, can read every Secret in it, including ones they did not create. Treat "who can get secrets in this namespace" as the actual blast radius of every Secret in it.

Application-layer secrets encryption encrypts Secret objects in etcd with a Cloud KMS key you control, so a raw etcd snapshot no longer discloses Secret contents in the clear:

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --database-encryption-key=projects/rc-saas-shared-sec-01/locations/\
us-central1/keyRings/kr-us-central1-gke/cryptoKeys/k-gke-etcd

The GKE service agent needs roles/cloudkms.cryptoKeyEncrypterDecrypter on that key (§14.8); this does not change how a Secret looks from inside the cluster, only how it is protected on disk.

The two lower-friction paths, in order of preference:

1. Workload Identity plus a client library call to Secret Manager directly from the application — no CSI driver, no cluster add-on, the fewest moving parts, and the pattern this book recommends by default.
2. The Secret Manager CSI driver (--enable-secret-manager), which mounts named secrets as files via a SecretProviderClass, authenticating through Workload Identity Federation for GKE — useful for workloads that expect a file-based secret and cannot be changed to call an API.

Pitfall. A ConfigMap holding a credential (§9.18) is the more common failure than either of these — it has no encryption story at all, because it was never a secret store.

9.18 ConfigMaps §

A ConfigMap is not a secret store — it is plain-text, world-readable to anyone with get on ConfigMaps in the namespace, and stored in etcd unencrypted regardless of whether database encryption (§9.17) is configured, because that setting applies only to the Secret resource type. Putting a database password, API key, or certificate private key in a ConfigMap is the finding, not a stylistic preference — the object type itself signals to every reviewer and every RBAC grant that the content is non-sensitive.

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: billing
  name: app-config
data:
  LOG_LEVEL: "info"
  FEATURE_FLAGS: "new-billing-ui=true"

Pitfall. Grep every ConfigMap in a cluster you inherit for anything that looks like a credential before trusting the cluster's Secret inventory — teams under deadline pressure reach for whichever object is already mounted.

9.19 Persistent Storage §

GKE mounts Persistent Volumes backed by the Compute Engine Persistent Disk CSI driver (regional or zonal, standard, balanced, SSD, or Extreme), Filestore for shared ReadWriteMany volumes, or Cloud Storage FUSE for object-backed access. Provisioning is dynamic through a StorageClass; the driver creates and attaches the underlying disk to match a PersistentVolumeClaim.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: sc-balanced-encrypted
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-balanced
  disk-encryption-kms-key: projects/rc-saas-shared-sec-01/locations/\
us-central1/keyRings/kr-us-central1-gke/cryptoKeys/k-gke-disks

Cloud SQL connectivity for stateful application data goes through the Cloud SQL Auth Proxy, whose IAM and authentication model is owned by §12.17; this chapter's concern is only the sidecar or CSI-driver pattern for running it inside a pod, which §12.17 covers directly.

Pitfall. A PersistentVolume's reclaim policy defaults to Delete for dynamically provisioned volumes — deleting the claim deletes the disk, including any data on it, unless the StorageClass sets reclaimPolicy: Retain.

9.20 Stateful Workloads §

StatefulSet gives pods stable network identity and stable storage across rescheduling — pod web-0 keeps its name, its DNS entry, and its bound PersistentVolumeClaim even after being rescheduled to a different node, which Deployment does not guarantee. This matters for anything that tracks peer identity itself: replicated databases, Kafka brokers, Elasticsearch nodes.

Security posture. StatefulSet pods are ordinary pods for RBAC, NetworkPolicy, and Pod Security Admission purposes — none of those controls loosen because a workload is stateful, and the durable storage a StatefulSet accumulates is a reason to be more careful about disk encryption (§9.19) and backup scope (§9.40), not less.

Pitfall. Scaling a StatefulSet down does not delete its PersistentVolumeClaims by default, so a scale-to-zero-and-back cycle reattaches old data — verify that is the intended behavior before relying on scale-down as a reset.

9.21 Ingress §

The GKE Ingress controller watches Ingress objects and provisions a Cloud Application Load Balancer to match, using container-native load balancing (§9.23) to route directly to pod IPs. It is the older, HTTP-only API; the Gateway API (§9.22) is the current recommendation for new workloads and adds protocol and cross-namespace routing Ingress never supported.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: billing
  name: billing-ingress
  annotations:
    kubernetes.io/ingress.class: "gce"
    networking.gke.io/managed-certificates: "billing-cert"
spec:
  rules:
    - host: billing.rickcollette.domain
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: billing-svc
                port:
                  number: 443

Pitfall. An Ingress with no explicit BackendConfig (§9.23) gets the load balancer's default health check and timeout behavior, which rarely matches an application's actual readiness signal — review it rather than accepting the default silently.

9.22 Gateway API §

The Gateway API is the upstream-standard, protocol-agnostic successor to Ingress: a GatewayClass and Gateway define the load balancer infrastructure, and one or more HTTPRoute, TCPRoute, or TLSRoute objects attach to it, including across namespaces — a capability Ingress never had, and one that matters for a platform team that owns the Gateway while application teams own their own HTTPRoute.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --gateway-api=standard

Security posture. Cross-namespace routing is opt-in per route through a ReferenceGrant, so a namespace cannot silently attach itself to another team's Gateway — the default is isolation, with attachment requiring explicit consent from the target namespace.

Pitfall. gcloud container clusters update --gateway-api requires the cluster to already be VPC-native and, for most Gateway classes, to be a Standard or Autopilot cluster on a sufficiently current version — check gcloud container get-server-config for the minimum version before committing a migration plan.

9.23 GKE Load Balancing §

Chapter 7 owns load balancer architecture; this section covers only how a GKE Service or Ingress binds to it. Container-native load balancing uses zonal Network Endpoint Groups (NEGs) that point directly at pod IPs instead of at node IPs and a NodePort — the endpoints in the NEG stay synchronized with the Service's actual pod membership, and it has been the default for GKE 1.17 and later. The cloud.google.com/neg annotation requests it explicitly where a Service needs to opt in outside the Ingress-managed default path:

apiVersion: v1
kind: Service
metadata:
  namespace: billing
  name: billing-svc
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
spec:
  selector:
    app: billing
  ports:
    - port: 443
      targetPort: 8443

The BackendConfig CRD attaches load balancer configuration — timeouts, Cloud CDN, Cloud Armor policy, Identity-Aware Proxy — to a Service via the cloud.google.com/backend-config annotation, and is where §7.6's backend service configuration becomes reachable from a GKE manifest at all; a Service with no BackendConfig gets the load balancer's defaults.

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  namespace: billing
  name: billing-backendconfig
spec:
  timeoutSec: 30
  healthCheck:
    checkIntervalSec: 15
    port: 8443
    type: HTTPS

Pitfall. A BackendConfig referenced by a Service annotation but with a typo in its name fails silently — the load balancer keeps its previous configuration rather than erroring, so verify propagation with gcloud compute backend-services describe rather than trusting the manifest alone.

9.24 Cloud Armor with GKE §

Chapter 7 owns Cloud Armor policy authoring (§7.13); binding one to a GKE workload is a BackendConfig field naming the policy, applied to the Service's backend service once container-native load balancing (§9.23) is active.

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  namespace: billing
  name: billing-backendconfig
spec:
  securityPolicy:
    name: "billing-edge-policy"

Security posture. The binding is per-Service, not per-cluster or per-namespace — a cluster with ten internet-facing Services needs ten BackendConfig reviews, and a Service added later with no BackendConfig ships with no Cloud Armor policy attached at all, silently.

Pitfall. Cloud Armor only inspects traffic through the external Application Load Balancer path; a Service exposed as LoadBalancer with an internal or passthrough network load balancer instead never reaches Cloud Armor regardless of policy configuration (§7.1).

9.25 Container Registry Integration §

The product formerly called Container Registry (gcr.io) is deprecated; Artifact Registry is its replacement and the only registry this book uses for GKE image pulls. A node's identity, not a Kubernetes-level credential, authorizes the pull: the node service account needs roles/artifactregistry.reader on the repository, and every image reference in a manifest uses the REGION-docker.pkg.dev host.

spec:
  containers:
    - name: billing
      image: us-central1-docker.pkg.dev/rc-saas-shared-art-01/\
app-images/billing:1.4.2

gcloud artifacts repositories add-iam-policy-binding app-images \
  --project=rc-saas-shared-art-01 \
  --location=us-central1 \
  --member="serviceAccount:sa-gke-node-rc-saas-prod-gke-01@\
rc-saas-prod-app-01.iam.gserviceaccount.com" \
  --role="roles/artifactregistry.reader"

Cross-project pulls are the normal case in this book's layout — images live in the shared artifact project rc-saas-shared-art-01 and are pulled by clusters in rc-saas-prod-app-01 — and need nothing beyond the IAM binding above; no registry-side network path or VPC peering is required because Artifact Registry is a regional Google API, reachable the same way any other Google API is (§5.17).

Remote and virtual repositories let a cluster pull from an upstream public registry (a remote repository proxying, for example, Docker Hub) through Artifact Registry's own IAM and caching rather than reaching the public internet directly, or present several repositories under one URL (a virtual repository) — both reduce the node's need for unrestricted outbound access to arbitrary registries.

9.26 Artifact Registry §

Artifact Registry is the registry itself — repository formats, encryption, retention, and scanning — as distinct from §9.25's GKE consumption of it.

gcloud artifacts repositories create app-images \
  --project=rc-saas-shared-art-01 \
  --location=us-central1 \
  --repository-format=docker \
  --kms-key=projects/rc-saas-shared-sec-01/locations/us-central1/\
keyRings/kr-us-central1-art/cryptoKeys/k-art-images \
  --immutable-tags

Security posture. --immutable-tags stops a tag such as :1.4.2 from being silently repointed at a different image digest after deployment, closing a supply chain path where a compromised CI pipeline republishes a tag without changing its name. --kms-key applies CMEK (§14.8) to stored artifacts; the repository's own service agent needs roles/cloudkms.cryptoKeyEncrypterDecrypter on that key.

Cleanup policies expire untagged or old images automatically, bounding both storage cost and the number of stale images an attacker could still pull if a tag were ever repointed to one.

Vulnerability scanning runs automatically against pushed images and surfaces findings against known CVEs; treat it as a detective control layered under Binary Authorization's preventive one (§9.27), not a substitute for it — scanning tells you what is already running, Binary Authorization decides what is allowed to start.

locals {
  art_kms_key = join("/", [
    "projects/rc-saas-shared-sec-01/locations/us-central1",
    "keyRings/kr-us-central1-art/cryptoKeys/k-art-images",
  ])
}

resource "google_artifact_registry_repository" "app_images" {
  project       = "rc-saas-shared-art-01"
  location      = "us-central1"
  repository_id = "app-images"
  format        = "DOCKER"
  kms_key_name  = local.art_kms_key
}

9.27 Binary Authorization §

Chapter 37 owns attestation authoring and the build-side chain of custody; this section covers only GKE's enforcement of a Binary Authorization policy. The policy is a project-level resource naming which attestors must have signed an image's digest before the cluster will admit it, evaluated at pod creation time by an admission controller.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --binauthz-evaluation-mode=project-singleton-policy-enforce

--enable-binauthz is deprecated in favor of --binauthz-evaluation-mode, whose only accepted values are disabled and project-singleton-policy-enforce. constraints/container.managed.enableBinaryAuthorization (§2.30) requires evaluation to be on org-wide.

Attestors are project-level resources referencing a Container Analysis note and a set of public keys; a policy requires one or more named attestors to have signed before admitting an image, and Chapter 37 covers how the signature is produced during the build.

Breakglass lets an operator bypass the policy for one deployment during an incident, via a signed annotation on the pod spec rather than a policy change — it is audited and time-bounded by convention, and using it should itself trigger an incident-review action item.

Dry-run mode logs what the policy would have blocked without blocking it, the correct way to validate a new policy against real traffic before switching to enforcement — roll it out per cluster, starting with the lowest-traffic environment.

9.28 Software Supply Chain Security §

GKE's software supply chain enforcement is the composition of three controls already introduced separately: Artifact Registry's immutable tags and vulnerability scanning (§9.26) at the storage layer, Binary Authorization (§9.27) at the admission layer, and Pod Security Admission (§9.14) at the runtime-configuration layer. None of them individually stops a compromised build pipeline from producing a signed, scannable, policy-compliant image that is nonetheless malicious — that is why Chapter 37 owns provenance and source integrity as a separate, earlier stage of the same chain.

What GKE's layer catches, and what it does not. Binary Authorization verifies an attestor signed a specific image digest; it says nothing about whether the attestor's own signing process was itself compromised, which is a build-system control (§37.3–§37.5), not a cluster-admission one. Vulnerability scanning finds known CVEs in installed packages; it does not find a deliberately introduced backdoor with no matching CVE.

Pitfall. Treating a green Binary Authorization check as proof an image is safe conflates "policy-compliant" with "trustworthy" — the policy is only as strong as the attestors behind it and the build process that produced their signatures.

9.29 GKE Security Posture §

The security posture dashboard is GKE's own aggregated findings surface — misconfiguration checks, workload vulnerability results, and (in enterprise mode) more advanced insights — separate from Security Command Center (Chapter 16) but feeding into it.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --security-posture=standard \
  --workload-vulnerability-scanning=standard

Both flags accept disabled, standard, or enterprise — enterprise mode adds deeper insights and, per the GKE Enterprise licensing note in §9.38, is billed under the GKE Enterprise edition rather than included at no charge.

Security posture. standard mode is a reasonable default for every cluster in an estate: it costs nothing beyond baseline GKE pricing and surfaces the same class of finding — missing NetworkPolicy, containers running as root, exposed dashboards — that a manual review would otherwise have to catch by hand.

Pitfall. The dashboard reports findings; it does not remediate them, and a cluster with a long-standing list of unaddressed findings provides no more security than a cluster with the dashboard turned off.

9.30 Cluster Hardening §

Cluster hardening is the composite of the individual controls in this chapter, applied as a deliberate baseline rather than discovered incident-by-incident. Google Cloud's own hardening guidance recommends placing teams and environments in separate namespaces or clusters, and reserving GKE Sandbox for untrusted or high-risk workloads and Confidential GKE Nodes for workload data that must stay encrypted in use — both signals that namespace-level isolation, while cheaper, is not equivalent to cluster-level or hardware-level isolation for genuinely adversarial tenants.

The baseline this book applies to every new cluster:

• Private nodes (§9.9), a DNS-based control plane endpoint with authorized networks enforced (§9.4, §9.10).
• VPC-native networking sized from the fixed CIDR plan (§9.7).
• Workload Identity Federation for GKE, with GKE_METADATA on every node pool (§9.11).
• Shielded Nodes and a dedicated minimal node service account (§9.31).
• Dataplane V2 for always-on NetworkPolicy (§9.16), with a default-deny policy per namespace (§9.15).
• Pod Security Admission at restricted for every namespace without a documented exception (§9.14).
• Binary Authorization in enforcing mode (§9.27), Artifact Registry immutable tags (§9.26).
• Application-layer Secret encryption with a Cloud KMS key (§9.17).
• Cloud Logging and Cloud Monitoring enabled with audit-relevant log types on (§9.33).

Organization policy applies the baseline, not just a checklist. constraints/container.managed.enablePrivateNodes, constraints/container.managed.enableShieldedNodes, constraints/container.managed.enableWorkloadIdentityFederation, constraints/container.managed.enableNetworkPolicy, constraints/container.managed.enableSecretsEncryption, constraints/container.managed.enableBinaryAuthorization, constraints/container.managed.disallowDefaultComputeServiceAccount, and constraints/container.managed.disableInsecureKubeletReadOnlyPort (§2.30) turn every item above from a convention a team can forget into a constraint the API itself enforces before cluster creation succeeds.

Pitfall. A hardening checklist applied once at cluster creation drifts: a node pool added six months later by a different engineer, using an older runbook, easily misses --workload-metadata=GKE_METADATA or --enable-shielded-nodes unless organization policy — not documentation — is what actually blocks the gap.

9.31 Node Hardening §

Chapter 8 owns the VM primitive underneath every node; this section covers only the GKE-specific surface layered on top of it. §8.16 owns Shielded VM mechanics and §8.29 owns instance service account design — GKE nodes apply both, plus GKE-specific defaults neither section covers.

Container-Optimized OS is GKE's default and recommended node image: a minimal, Google-maintained, auto-patching OS with a read-only root filesystem and no package manager, which removes most of the persistence mechanisms a compromised node would otherwise offer an attacker.

Shielded Nodes are a cluster-wide setting, turned on once at cluster creation; Secure Boot and Integrity Monitoring are then configured per node pool:

gcloud container clusters create rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --enable-shielded-nodes

gcloud container node-pools create np-default \
  --cluster=rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --shielded-secure-boot \
  --shielded-integrity-monitoring \
  --metadata=disable-legacy-endpoints=true \
  --service-account=sa-gke-node-rc-saas-prod-gke-01@\
rc-saas-prod-app-01.iam.gserviceaccount.com \
  --enable-autoupgrade \
  --enable-autorepair

Shielded Nodes (§8.16) let the control plane cryptographically verify that every node is a genuine VM in the cluster's own managed instance group; Secure Boot additionally blocks unsigned kernel modules and is off by default, worth enabling explicitly on Ubuntu node images; Integrity Monitoring is on by default and watches for rootkit and bootkit tampering. --metadata=disable-legacy-endpoints=true closes the unauthenticated v1beta1 metadata API some older container images still probe for.

A dedicated minimal node service account — sa-gke-node-<cluster>, never the default Compute Engine service account (constraints/container.managed.disallowDefaultComputeServiceAccount, §2.30) — holds only roles/logging.logWriter, roles/monitoring.metricWriter, roles/stackdriver.resourceMetadata.writer, and roles/artifactregistry.reader; any additional access an application needs comes through Workload Identity Federation (§9.11), never through the node identity.

Node auto-upgrade (--enable-autoupgrade) keeps nodes on a patched kubelet and OS image aligned to the cluster's release channel (§9.35); leaving it off is the single most common way a fleet accumulates known-vulnerable node images.

Autopilot removes nearly all of this section's decisions: Container- Optimized OS, Shielded Nodes, a minimal service account, and auto-upgrade are all mandatory and non-configurable, which is the concrete meaning of "Autopilot narrows the node-level attack surface" from §9.3.

9.32 Runtime Security §

Runtime security is what happens after a container has started and is executing — a layer beneath admission-time controls (Pod Security Admission, Binary Authorization) that catches behavior no static policy predicted. GKE's own tooling here is the security posture dashboard's runtime findings (§9.29) and integration with third-party runtime detection agents (typically a DaemonSet watching syscalls or eBPF-visible behavior), rather than a single first-party runtime IDS.

GKE Sandbox (§9.2) and Confidential GKE Nodes are the two infrastructure-level runtime controls worth calling out again here: sandboxing narrows what a compromised container can do to the kernel, and Confidential GKE Nodes encrypt workload memory in use, protecting against a host-level compromise or a malicious co-tenant with hypervisor access — neither replaces least-privilege RBAC and NetworkPolicy, both raise the cost of a successful container breakout.

Pitfall. Runtime detection generates alerts, not blocks, in most deployments — pair it with an on-call process that actually reads the alerts, or it is indistinguishable from having no runtime security at all.

9.33 Logging and Monitoring §

GKE integrates with Cloud Logging and Cloud Monitoring at the cluster level, controlled by two independent flags accepting overlapping but distinct component lists.

gcloud container clusters create rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --logging=SYSTEM,WORKLOADS \
  --monitoring=SYSTEM,WORKLOADS \
  --enable-managed-prometheus

Security posture. SYSTEM logging (control plane, kubelet, and system-component logs) is the minimum for any incident investigation; WORKLOADS adds container stdout/stderr, and API server audit logs are enabled separately as part of Cloud Audit Logs (§3.x) rather than through this flag. Google Cloud Managed Service for Prometheus (--enable-managed-prometheus) is this book's default for workload metrics — it removes the operational burden of running and securing self-managed Prometheus inside the cluster.

Pitfall. --logging=NONE or --monitoring=NONE are valid values that some cost-cutting exercises reach for — the resulting cluster has no audit trail for workload behavior at all, which is a materially different security posture than "logging enabled but not yet reviewed."

9.34 Cluster Upgrades §

GKE upgrades the control plane automatically on a schedule tied to the cluster's release channel (§9.35); node pool upgrades follow the same channel unless auto-upgrade is disabled, in which case they require explicit action and quietly fall behind.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --maintenance-window=03:00

Rollout controls. --max-surge-upgrade and its companion surge settings bound how many extra nodes a rolling node pool upgrade creates at once, trading upgrade speed against capacity headroom; maintenance exclusions (--add-maintenance-exclusion-*) block upgrades during a defined window, the correct tool for a freeze period rather than disabling auto-upgrade outright.

Security posture. A control plane more than one minor version behind the latest in its channel is increasingly likely to be missing a security patch that has already shipped upstream — auto-upgrade with a well-chosen maintenance window is materially safer than manual upgrades most estates actually perform on schedule.

Pitfall. Disabling node auto-upgrade to avoid a maintenance-window disruption is a common trade that quietly reintroduces every node-level CVE patched since the last manual upgrade — prefer a maintenance exclusion over disabling auto-upgrade.

9.35 Release Channels §

A release channel decides how quickly a cluster receives new Kubernetes minor versions and patches, trading feature currency against stability. Rapid ships a new minor version one to two weeks after upstream GA; Regular, the default, arrives about two months after Rapid; Stable arrives three to four months after Regular, after the most external validation; Extended holds a minor version for up to 24 months total — roughly 14 months of standard support plus about 10 months of extended security patches.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --release-channel=regular

Not enrolling in a release channel at all is deprecated and scheduled for removal on 2027-06-14; every new cluster should specify one explicitly.

Pitfall. Extended's 24-month ceiling is a maximum, not a promise of inaction — a cluster left on Extended without a deliberate upgrade plan still eventually falls off supported versions and loses security patches entirely.

9.36 Multi-Cluster Architecture §

Running more than one GKE cluster is the default at any meaningful scale — for blast-radius isolation between environments, for regional failover, or because a single cluster's control plane and node-pool limits are a ceiling worth avoiding well before hitting it. Multi-Cluster Ingress is the cloud-hosted controller that gives clusters in different regions a single, consistent virtual IP for one logical application, routing to whichever cluster is closest and healthy — the mechanism this book uses for the active-active production topology spanning us-central1 and europe-west1.

Security posture. Multi-cluster designs multiply the attack surface of every per-cluster control in this chapter — private nodes, Workload Identity, Binary Authorization policy — by the cluster count, unless that configuration is itself managed centrally, which is exactly the problem Fleet management (§9.37) and Config Sync (§9.38) exist to solve.

Pitfall. A security control applied to one cluster in a multi-cluster estate and forgotten on a second, newer cluster is the most common multi-cluster security gap, and it is invisible until an audit compares clusters directly rather than reviewing each in isolation.

9.37 Fleet Management §

A fleet is a set of registered clusters — GKE, on-premises, or other Kubernetes — under one logical umbrella for applying features consistently. Registering a cluster to a fleet is the prerequisite for Multi-Cluster Ingress, Multi-Cluster Services, Config Sync, and Policy Controller alike.

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --enable-fleet

roles/gkehub.admin administers fleet membership and features; roles/gkehub.viewer is read-only. Fleet-level RBAC through roles/gkehub.gatewayAdmin and roles/gkehub.gatewayReader governs the fleet's connect gateway, which proxies kubectl access to member clusters through a single, centrally authorized path instead of each cluster's own control plane endpoint.

Security posture. The connect gateway is worth preferring over direct control-plane access for private clusters reached from outside the VPC — it removes the need for authorized networks entries per operator location, substituting an IAM-gated proxy Google operates.

Pitfall. Fleet membership does not itself change any per-cluster security setting — it is an organizing layer, and the actual configuration consistency across member clusters still depends on Config Sync (§9.38) or an equivalent GitOps pipeline being applied to all of them.

9.38 Config Sync §

Config Sync is a GKE Enterprise feature: a GitOps controller that continuously reconciles cluster state — RBAC, NetworkPolicy, namespaces, any Kubernetes object — against a Git repository, so a fleet of clusters stays configured identically without a human running kubectl apply against each one by hand.

Installing Config Sync applies a ConfigManagement custom resource to the member cluster (via kubectl apply, since the fleet-level apply command is Preview/Beta and must not be a production dependency); once installed, its status is queried at the fleet level:

gcloud container fleet config-management describe \
  --project=rc-saas-prod-app-01

This book prints no price for it. GKE's pricing page is the authority and it changes; a per-vCPU-hour figure copied into a book is wrong by the time it is read, so check cloud.google.com/kubernetes-engine/pricing for the current GKE Enterprise rate before budgeting.

Licensing implication. Config Sync ships as part of the GKE Enterprise edition rather than GKE Standard's included feature set — budget it as a metered addition to a fleet's cost, not an included capability, and confirm current pricing against the GKE pricing page before committing an estate to it.

Security posture. The Git repository Config Sync reads from becomes a privileged path into every member cluster's configuration — protect it with the same branch-protection and review discipline as the application source repositories it deploys alongside (Chapter 37), because a compromised commit here can push a permissive RBAC binding to every cluster in the fleet at once.

9.39 Policy Controller §

Policy Controller is also a GKE Enterprise feature: a validating and mutating admission controller built on the open source Open Policy Agent Gatekeeper project, enforcing constraint templates written in Rego across every cluster in a fleet from one set of policy bundles.

gcloud container fleet policycontroller enable \
  --memberships=rc-saas-prod-gke-01 \
  --location=us-central1

Security posture. Policy Controller is the fleet-wide equivalent of the per-cluster admission controls in this chapter — Pod Security Admission (§9.14), Binary Authorization (§9.27) — generalized to arbitrary policy (mandatory labels, allowed registries, resource limits) and applied consistently everywhere a cluster joins the fleet, rather than configured per cluster.

Licensing implication. Like Config Sync, Policy Controller requires GKE Enterprise edition; an estate running only GKE Standard clusters can still use Pod Security Admission and Binary Authorization per cluster but cannot enforce a shared policy bundle across clusters without upgrading the fleet's edition.

Pitfall. Constraint templates in dryrun enforcement action log violations without blocking them — the same rollout discipline as Binary Authorization's dry-run mode (§9.27) applies here, and skipping it risks a fleet-wide outage from an overly strict constraint pushed straight to enforcement.

9.40 GKE Backup §

GKE Backup is a GA, managed backup service for GKE workloads — application manifests, persistent volume data, and (optionally) Secrets — organized into backup plans that run on a schedule against a defined scope (namespaces, label selectors, or the whole cluster) and land in a backup vault. Enable the add-on on the cluster, then define the plan as code:

gcloud container clusters update rc-saas-prod-gke-01 \
  --project=rc-saas-prod-app-01 \
  --region=us-central1 \
  --update-addons=BackupRestore=ENABLED

locals {
  billing_cluster_id = join("/", [
    "projects/rc-saas-prod-app-01/locations/us-central1",
    "clusters/rc-saas-prod-gke-01",
  ])
}

resource "google_gke_backup_backup_plan" "billing" {
  name     = "billing-backup-plan"
  project  = "rc-saas-prod-app-01"
  location = "us-central1"
  cluster  = local.billing_cluster_id

  backup_config {
    include_volume_data = true
    include_secrets     = false
    selected_namespaces {
      namespaces = ["billing"]
    }
  }

  backup_schedule {
    cron_schedule = "0 3 * * *"
  }

  retention_policy {
    backup_retain_days = 30
  }
}

roles/gkebackup.backupAdmin manages plans and vaults; roles/gkebackup.viewer is read-only; roles/gkebackup.restoreAdmin is the separate, narrower role for performing a restore, and should not be held by the same operators who manage the backup schedule by default.

Security posture. A backup vault should live in the security project (rc-saas-shared-sec-01, §2.21) rather than the workload project, so that a compromise of the production project's IAM does not automatically grant the attacker delete access to its own backups.

Pitfall. A backup plan scoped by namespace silently stops covering a new namespace added after the plan was created — review plan scope whenever a cluster's namespace layout changes, rather than assuming "we back up this cluster" covers everything in it.

9.41 Disaster Recovery §

GKE disaster recovery composes GKE Backup (§9.40) for data and configuration, Multi-Cluster Ingress (§9.36) for traffic failover, and the cluster-as-code definition (Terraform) for recreating a cluster's shape in a second region — no single GKE feature is "the DR feature."

The recovery sequence: recreate the cluster from its Terraform definition in the target region, restore workloads and volumes from the most recent GKE Backup vault snapshot, then shift Multi-Cluster Ingress traffic weight once health checks pass.

Pitfall. A recovery plan tested only against the primary region's cluster configuration, never actually executed against a second region, routinely discovers region-specific gaps — a missing secondary range, an unregistered fleet membership — only during a real outage, which is the worst possible time to discover them.

9.42 GKE for SaaS §

A multi-tenant SaaS platform on GKE has to decide, per tenant tier, how much isolation a tenant's workload gets: shared namespace with RBAC and NetworkPolicy boundaries (cheapest, weakest isolation), dedicated namespace per tenant on shared nodes, or dedicated node pool — even a dedicated cluster — for tenants whose contract or regulatory posture demands it.

This book's SaaS reference pattern: rc-saas-prod-gke-01 runs platform-shared services (the application itself, its supporting services) in namespaces matching the application slug, with Workload Identity Federation scoping each service's GCP access narrowly, Pod Security Admission at restricted, and NetworkPolicy default-deny per namespace — tenant data isolation is enforced at the application and database layer (row-level or schema-level), not by giving every tenant a Kubernetes namespace, because namespace count at SaaS scale becomes an operational burden of its own well before it becomes a meaningful security boundary.

Pitfall. Multi-tenancy decisions made implicitly — "we'll just put everyone in the same namespace for now" — are expensive to reverse once a tenant's contract or a compliance requirement demands the isolation the initial design never planned for; decide the isolation tier before the first paying tenant with elevated requirements arrives, not after.

9.43 GKE for Enterprise Applications §

An enterprise estate running GKE typically starts from stricter defaults than a SaaS estate: on-premises connectivity through Interconnect or VPN (Chapter 6) reaching the cluster's private control plane and nodes, multiple business-unit folders each owning their own clusters under a shared organization policy baseline, and a heavier weighting toward GKE Enterprise for fleet-wide Config Sync (§9.38) and Policy Controller (§9.39) because a multi-business-unit estate is exactly the scenario those features are priced for — dozens of clusters that must provably run the same policy bundle, not a handful an operator can review by hand.

The enterprise reference pattern: clusters named rc-ent-<env>-gke-<nn> per business unit, each private with a DNS-based control plane endpoint and authorized networks scoped to the corporate WAN range reached through Interconnect, Workload Identity Federation bound to IAM groups synced from the enterprise directory rather than individually provisioned Google service accounts, Binary Authorization enforcing against attestors tied to the enterprise's own build pipeline (Chapter 37), and Config Sync reconciling every cluster's baseline RBAC and NetworkPolicy from a single reviewed Git repository.

What differs from SaaS, concretely: more clusters, more folders, and proportionally more reliance on fleet-wide tooling (§9.37–§9.39) rather than per-cluster review, because per-cluster review does not scale past a handful of clusters and an enterprise estate typically exceeds that count within its first year.

Pitfall. Applying the SaaS estate's single-cluster mental model to an enterprise rollout — reviewing each cluster's Binary Authorization policy or RBAC bindings by hand — is the pattern that breaks first as cluster count grows; invest in Config Sync and Policy Controller before the manual process becomes the bottleneck, not after.

Chapter Summary §

• GKE separates control-plane ownership (Google-managed, in a tenant project) from node ownership (your project, your service account, your patch cadence); Autopilot narrows the node-level decisions to almost none.
• The control plane now offers a DNS-based endpoint (--enable-dns-access, recommended) and an IP-based endpoint (--enable-ip-access); authorized networks must be separately enforced on the DNS endpoint with --enable-authorized-networks-on-private-endpoint.
• VPC-native networking is the only mode; the pod secondary range's size, divided by --default-max-pods-per-node, is what actually bounds cluster node capacity, and a secondary range in use cannot be removed.
• §4.12 owns Workload Identity Federation for GKE's identity mechanics; §9.11 owns only --workload-pool and --workload-metadata=GKE_METADATA, and a node pool left on legacy metadata mode defeats the pool for every pod scheduled to it.
• Kubernetes Secrets are base64, not encrypted, and world-readable to anyone with get secrets in the namespace; a ConfigMap is never an acceptable substitute for a secret, and neither is a substitute for Secret Manager (Chapter 13) plus Workload Identity as the default pattern.
• PodSecurityPolicy was removed in Kubernetes 1.25; Pod Security Admission and the Pod Security Standards (privileged, baseline, restricted) are the replacement, set per namespace via labels.
• Dataplane V2 makes Kubernetes NetworkPolicy always-on; without it, network policy enforcement is a separate flag that is easy to leave off.
• Binary Authorization enforcement is --binauthz-evaluation-mode=project-\ singleton-policy-enforce; the deprecated --enable-binauthz should not appear in new configuration.
• Container Registry (gcr.io) is deprecated; every image reference in this book uses Artifact Registry's REGION-docker.pkg.dev, authorized through the node service account's roles/artifactregistry.reader.
• Node hardening is Container-Optimized OS, Shielded Nodes, a dedicated minimal service account, and auto-upgrade; Autopilot makes all of it non-configurable and mandatory.
• Config Sync and Policy Controller are GKE Enterprise features, billed separately from GKE Standard, and are the correct tool once fleet size makes per-cluster review impractical.
• GKE Backup composes with Multi-Cluster Ingress and cluster-as-code Terraform to form disaster recovery; no single feature is "the DR feature," and an untested recovery plan reliably has gaps.
• The container.managed family of organization policy constraints (§2.30, §9.30) turns this chapter's hardening baseline from a convention into an enforced default at cluster-creation time.

Security Checklist §

Sources §

• Types of GKE clusters (Autopilot vs. Standard) — https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters (last validated 2026-09-03)
• Private cluster concepts and control plane endpoint model — https://cloud.google.com/kubernetes-engine/docs/concepts/private-cluster-concept (last validated 2026-09-03)
• Workload Identity Federation for GKE — https://cloud.google.com/kubernetes-engine/docs/concepts/workload-identity (last validated 2026-09-03)
• GKE Dataplane V2 — https://cloud.google.com/kubernetes-engine/docs/concepts/dataplane-v2 (last validated 2026-09-03)
• GKE release channels — https://cloud.google.com/kubernetes-engine/docs/concepts/release-channels (last validated 2026-09-03)
• Shielded GKE Nodes — https://cloud.google.com/kubernetes-engine/docs/how-to/shielded-gke-nodes (last validated 2026-09-03)
• Harden your GKE cluster's security — https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster (last validated 2026-09-03)
• Multi-Cluster Ingress — https://cloud.google.com/kubernetes-engine/docs/concepts/multi-cluster-ingress (last validated 2026-09-03)
• Container-native load balancing through NEGs — https://cloud.google.com/kubernetes-engine/docs/how-to/container-native-load-balancing (last validated 2026-09-03)
• Access scopes for node service accounts — https://cloud.google.com/kubernetes-engine/docs/how-to/access-scopes (last validated 2026-09-03)
• Storing container images in Artifact Registry — https://cloud.google.com/artifact-registry/docs/docker/store-docker-container-images (last validated 2026-09-03)
• GKE Sandbox overview — https://cloud.google.com/kubernetes-engine/docs/concepts/sandbox-pods (last validated 2026-09-03)
• Migrate from PodSecurityPolicy to PodSecurity — https://cloud.google.com/kubernetes-engine/docs/how-to/migrate-podsecuritypolicy (last validated 2026-09-03)
• BackendConfig CRD for Ingress and Cloud Armor — https://cloud.google.com/kubernetes-engine/docs/concepts/backendconfig (last validated 2026-09-03)
• Secret Manager add-on / CSI driver for GKE — https://cloud.google.com/secret-manager/docs/secret-manager-managed-csi-component (last validated 2026-09-03)
• GKE pricing (editions and metered features) — https://cloud.google.com/kubernetes-engine/pricing (last validated 2026-09-03)
• Cloud SDK command surface resolved offline against Google Cloud SDK 583.0.0, 2026-09-03
• hashicorp/google provider resource schema for google_container_cluster, google_container_node_pool, google_artifact_registry_repository — https://raw.githubusercontent.com/hashicorp/terraform-provider-google/main/website/docs/r/container_cluster.html.markdown (provider 8.x) (last validated 2026-09-03)