Chapter 28

Ansible on GCP

Scope. This chapter owns Ansible as the book's configuration-management tool: what it is for on Google Cloud and what it is not for, the dynamic inventory, keyless authentication, in-guest configuration, secrets, roles and collections, idempotency, and the seam with Terraform. It does not re-teach the services Terraform provisions (Chapter 26), OS Login (§8.19), Secret Manager (Chapter 13), or the OS hardening target state (§32.1). Long playbooks and inventory files live in code/ansible/. Prerequisites. Chapter 26 (Terraform), Chapter 8 (§8.19 OS Login, §8.20 SSH Access, §8.21 IAP TCP forwarding), Chapter 13 (Secret Manager), Chapter 4 (§4.3 Workload Identity Federation, §4.16 Identity-Aware Proxy). Verified against. ansible-core 2.21.3 with the google.cloud collection 1.14.0, exercised directly with ansible-doc; Cloud SDK 583.0.0; see sources at end.

Ansible's position on Google Cloud is narrower than its module count suggests, and being honest about the boundary is what makes it useful. The google.cloud collection ships 194 modules that create GCP resources, and this book uses almost none of them. It uses the inventory plugin, the Secret Manager lookup, the IAP connection plugin, and the core modules that configure the inside of a machine.

The reason is not preference, and it is not a state-file argument. It is that no resource-managing module in the collection supports check mode. All 79 modules that declare it are *_info modules; the roughly 115 that create or change infrastructure declare none. A module without check-mode support is skipped in check mode rather than simulated, so ansible-playbook --check over a provisioning play reports nothing and changes nothing. Ansible has no terraform plan for GCP resources. Every control in §26.23 and §26.24 — review the diff, approve the artifact, apply what was approved — has no equivalent, because there is no diff to review.

On the configuration side the situation inverts. template, copy, lineinfile, user, package, and systemd_service all support check mode and diff mode fully, so --check --diff on an in-guest role is genuinely informative. That asymmetry is the whole argument of §28.2 and it is the reason §28.22 is the chapter's most important section.

One more thing must be said plainly. The collection's own README recommends service account keys for unattended operation. This book does not, anywhere, and §28.6 is the answer. That disagreement is not a style difference — it is the single most consequential decision in the chapter.

28.1 Ansible’s Role in GCP §

Ansible's job on Google Cloud is the inside of a machine: packages, files, users, services, and kernel parameters on instances that something else created.

What it does well here:

What it should not do here is create GCP resources, and the collection's own gaps say so more clearly than any argument: there is no IAM binding module of any kind, no firewall policy module beyond the legacy gcp_compute_firewall, no organization policy module, no VPC Service Controls module, and no CMEK argument on any module — only customer-supplied raw_key. An estate built with Ansible cannot express the controls this book spends twenty-five chapters establishing.

Every module is generated and community-supported. The collection's modules carry status: preview and supported_by: community. That is a supply-chain and support fact worth knowing before a production dependency forms (§28.18).

Pitfall. The 194-module count invites the conclusion that Ansible is an alternative to Terraform on GCP. Count the modules you would actually need for a landing zone and the gaps above appear immediately — usually after the first two weeks of work.

28.2 Provisioning vs. Configuration Management §

The division is not philosophical. It follows from one measurable property: which tool can tell you what it is about to do.

Terraform can; Ansible cannot, for GCP resources.

So: Terraform owns resource existence and shape; Ansible owns what happens inside. The line is the instance boundary, and it is a clean one because the two tools' strengths sit on opposite sides of it.

This also settles who owns idempotency. Terraform's idempotency comes from state; Ansible's comes from each module comparing current state to declared state at run time (§28.19). Neither is better, but only one of them can be inspected before it runs.

The exception worth allowing is a resource with no Terraform representation and a real operational need — and even then, the honest treatment is a documented exception, not a second provisioning tool (§26.35's argument about local-exec applies for the same reason).

Pitfall. Teams discover the check-mode gap after building a provisioning playbook, usually during an incident when someone wants to know what a playbook will do before running it in production. The answer is that nobody can know, and by then the playbook is load-bearing.

28.3 Dynamic Inventory §

The google.cloud.gcp_compute inventory plugin queries the Compute Engine API and builds the inventory from what actually exists. It replaces a static host list, which is always wrong in an autoscaled estate.

The filename is a control, and it is the first thing that goes wrong. The plugin accepts exactly four suffixes — gcp.yml, gcp.yaml, gcp_compute.yml, gcp_compute.yaml. A file named anything else is silently ignored: the plugin returns false from its verification, and Ansible reports an empty inventory rather than an error.

The default hostnames is wrong for this book's estate. It is [public_ip, private_ip, name] — public IP first. Every instance here is created without an external address (§33.8), so the default produces hosts nobody can reach. Name the instance and set ansible_host explicitly:

plugin: google.cloud.gcp_compute
projects:
  - rc-saas-prod-app-01
zones: [us-central1-a, us-central1-b, us-central1-c]
auth_kind: machineaccount
filters:
  - 'status = RUNNING'
  - 'labels.env = prod'
hostnames: [name]
compose:
  ansible_host: networkInterfaces[0].networkIP
  ansible_gcloud_zone: zone
strict: true

The full inventory is code/ansible/inventory/prod.gcp_compute.yml.

filters uses the Compute Engine aggregatedList filter language, not Jinja, and multiple entries are ANDed. Naming zones matters: omitting it lists every zone in the project, which is slow and expensive on a large estate.

strict: true turns a bad compose or keyed_groups expression into an error instead of a silently missing group — which, since groups decide which hosts a play targets, is a safety control rather than a debugging convenience.

Verify the inventory before trusting it. ansible-inventory --graph and --list render what the plugin actually produced, and running it once after any change is how the silent-empty-inventory failure gets caught.

Pitfall. Using folders instead of projects makes the plugin call Resource Manager to enumerate projects, and it forcibly overrides scopes to cloud-platform for that call. The identity then needs resourcemanager.projects.list on the folder, and a Compute-only scope is not enough — which surfaces as a permission error naming an API nobody configured.

28.4 GCP Authentication §

Every google.cloud module and the inventory plugin take the same required auth_kind parameter, and it has exactly four values.

Three of the four are keyless. This matters because most published Ansible-on-GCP material assumes the fourth, and the book's position (§26.3, §33.5) is that a downloaded key is a permanent exfiltratable credential.

The credential parameters are mutually exclusive with the wrong auth_kind, and the failure is loud. Supplying service_account_file with anything but serviceaccount fails the task outright rather than falling back. That is a good property: a stray key path left in a group_vars file after switching to machineaccount is an error, not a silent regression.

service_account_email only means something with machineaccount, where it selects a non-default attached identity. It is not a way to name an account to impersonate — a genuinely misleading parameter name.

Eight environment variables back these parameters — GCP_PROJECT, GCP_AUTH_KIND, GCP_SERVICE_ACCOUNT_FILE, GCP_SERVICE_ACCOUNT_EMAIL, GCP_SERVICE_ACCOUNT_CONTENTS, GCP_ACCESS_TOKEN, GCP_SCOPES, and GCP_ENV_TYPE. Playbook values win; environment values are used only when the playbook does not set them.

Pitfall. auth_kind is required and the modules surface GCP_AUTH_KIND only in their notes, not in the documented options. A team that sets it in the environment and omits it from the inventory file has an inventory that works on one machine and fails on another, with an error about a missing required parameter rather than about the environment.

28.5 Service Account Authentication §

auth_kind: serviceaccount is the only value that consumes a key file, and this section exists to say why the book does not use it.

The collection recommends it and Google recommends against it. The google.cloud README states that for unattended operation it is common to use service account keys. Google's own credential guidance says to avoid downloaded keys, avoid putting them on the filesystem, and — notably — not to solve the problem by putting the key in a secret manager either, because that just moves it.

What a key actually costs, restated in Ansible's terms:

• It does not expire, so a leaked key is valid until someone notices and revokes it (§34.5 covers why revocation is slower than it looks).
• It lives on the controller's filesystem or in a variable, which means it is in whatever backs that — a Vault file, a CI secret store, a developer laptop.
• It is the same credential everywhere it is copied, so a controller compromise is a full compromise of everything that account can reach.
• constraints/iam.disableServiceAccountKeyCreation (§31.4) makes it unavailable in a hardened estate anyway.

service_account_contents is worse than service_account_file, not better. Passing the key material as a variable puts it in play output, in --verbose logs, and in any fact cache, in addition to wherever the variable came from.

If a key is genuinely unavoidable — an Ansible controller on hardware outside Google Cloud with no federation path — then scope it to one project, give it the narrowest role set, rotate it on a schedule you actually run, and record the exception. That is a managed risk. The default is §28.6.

Pitfall. Copying a key into a Vault-encrypted group_vars file feels like a control and is mostly a filing decision (§28.15). The key is still a non-expiring credential; it is now also in your repository's history.

28.6 Workload Identity Authentication §

This is the chapter's default. Both keyless paths are supported by the collection and neither requires a change to any module argument.

On a controller inside Google Cloud, use machineaccount:

auth_kind: machineaccount

The controller VM has an attached service account, the metadata server issues short-lived tokens, and nothing is stored. This is the whole configuration — the credential parameters are not used and must not be set (§28.4).

On a runner outside Google Cloud, use application:

auth_kind: application

application delegates to Application Default Credentials, and an external account credential configuration file produced by Workload Identity Federation is an ADC credential type. The federation itself is §4.3 and its CI wiring is §4.7 and §26.4 — unchanged here, because Ansible is simply another ADC consumer.

auth_kind: application accepts none of the credential parameters. All of its configuration is environmental: GOOGLE_APPLICATION_CREDENTIALS pointing at the credential configuration file, or an ambient ADC from gcloud auth application-default login for local work. There is no in-playbook argument for a federation file, and looking for one is a common wrong turn.

Grant the controller's identity the minimum. For the inventory plugin that is compute.instances.list on the project — read-only, and far less than the roles a provisioning playbook would need. This is a direct dividend of §28.2's division of labor.

Every call is attributable. The collection sets a Google-Ansible-MM-<product> user agent on every request, so Ansible's API activity is identifiable in audit logs by requestMetadata.callerSuppliedUserAgent (§28.21).

Pitfall. application on a Compute Engine VM also works, because ADC falls back to the metadata server. It is then indistinguishable from machineaccount in the playbook but resolves differently if GOOGLE_APPLICATION_CREDENTIALS is ever set in the environment. Say which one you mean.

28.7 Managing Compute Engine §

The collection can create instances. This book does not, and the section explains where the line falls rather than demonstrating the modules.

gcp_compute_instance covers the shape of an instance — machine type, disks, network_interfaces, service_accounts, metadata, scheduling — and Terraform covers the same ground with a plan you can review (§28.2). Where the two overlap, Terraform wins by default.

Where the collection simply cannot go:

Use the *_info modules freely. They support check mode, they read rather than write, and gcp_compute_instance_info with a filters expression is a good way for a playbook to make a decision about the fleet it is running against.

Where an instance really must be created from Ansible — a short-lived test fixture, a disposable analysis VM during an incident (§34.7) — the module is fine for that. Its weaknesses are all about long-lived, reviewed, drift-checked infrastructure, and a fixture is none of those things.

Pitfall. network_interfaces with an access_configs entry assigns an external IP, and the module's shape makes that look like a required part of the interface rather than an opt-in. Omit access_configs entirely; §33.8 is the failure it creates.

28.8 VM Configuration §

This is where Ansible earns its place: converging the inside of an instance that Terraform created.

A play targets a group the inventory built from a Terraform-set label (§28.22), so nothing is written down twice:

- name: Converge application hosts
  hosts: gcp_web
  become: true
  roles:
    - os-baseline
    - app-runtime

become: true is the privilege boundary and deserves attention. The SSH identity is a person or a pipeline; the escalated identity is root on the host. OS Login with roles/compute.osLogin versus roles/compute.osAdminLogin (§8.19) decides who can escalate at all, and that is the control — a become in a playbook is not.

Order matters more than it does in Terraform. Ansible executes tasks in written order with no dependency graph, so a role that writes a config file before installing the package that owns its directory fails. Handlers, notify, and serial are the ordering tools.

Converge, then assert. A role that only makes changes reports success when it ran, not when the host is correct. Adding read-back assertions (§28.19) turns a converged host into a verified one, which is the difference §29.11 cares about.

Pitfall. A play with hosts: all against a dynamic inventory targets whatever the API returned, which after an autoscaling event is not the set the author had in mind. Always target a group, and use --limit for anything ad hoc.

28.9 Package Management §

Package state is the most common thing a baseline role manages, and the portability question decides how it is written.

Use ansible.builtin.package for cross-distribution roles and the specific module — dnf, apt — when a distribution-specific option is needed. A golden image estate (§8.33) usually standardizes on one distribution, in which case the specific module is clearer.

Pin what matters and let the rest float. A security baseline usually wants state: present for tools it requires and state: absent for packages it forbids. state: latest on every package makes each run a potentially breaking change with no review, which is §26.35's unpinned-version anti-pattern in another form.

Package installation needs egress, and this estate denies it by default. Instances have no external IP and egress is deny-default (B01). Reaching a distribution mirror therefore requires Cloud NAT or, better, a private mirror inside the VPC. Discovering this during a playbook run is common; designing for it is not.

The stronger pattern is to bake, not converge. §8.33's golden images move package installation into an image build (Chapter 23) where it is scanned, versioned, and promoted. Ansible then enforces configuration rather than installing software, and a drifted package set becomes an image problem with an audit trail.

Pitfall. state: latest in a role that also reports changed on every run trains the team to ignore Ansible's change output, which is the signal §28.19 depends on.

28.10 User Management §

Local user management on GCP instances is mostly a question of what should not exist, because OS Login owns the accounts that should.

OS Login (§8.19) provisions Linux accounts from Google identities, keyed to IAM roles, with the account lifecycle following the identity. That removes the reason most local accounts exist and it removes the SSH key distribution problem entirely.

So the role's job is subtraction:

- name: Remove local accounts that OS Login replaces
  ansible.builtin.user:
    name: "{{ item }}"
    state: absent
    remove: true
  loop: "{{ os_baseline_absent_users }}"

A local account with an authorized key is a credential outside IAM. It survives the person leaving, it is invisible to an access review (§29.4), and it is not affected by revoking their Google identity. That is the failure; the module above is the fix.

Service accounts on the host are different. A daemon needing a system user — nginx, an application account — is legitimate, should have shell: /usr/sbin/nologin and no password, and should be created by the role that installs the daemon.

Pitfall. Managing authorized_keys with Ansible works and quietly recreates the problem OS Login solved: keys distributed by a playbook, revoked by another playbook run, with no connection to the IAM policy anyone audits. If OS Login is enabled, the role should be removing keys, not placing them.

28.11 SSH Hardening §

§8.19 owns OS Login and §8.20 owns SSH access to instances. This section owns the part Ansible is responsible for: the transport it uses, and the daemon configuration it enforces.

The transport question comes first, because this estate's instances have no external IP. The collection ships a first-party connection plugin for exactly this:

ansible_connection: google.cloud.iap

google.cloud.iap shells out to gcloud compute start-iap-tunnel and runs the standard SSH transport over the resulting local port. It needs the zone per host, which the inventory supplies through compose: ansible_gcloud_zone: zone (§28.3). This is better than hand-assembling ansible_ssh_common_args because it is maintained, and because the tunnel's lifecycle is the plugin's problem.

Two of its defaults want changing. private_key_file defaults to ~/.ssh/google_compute_engine, which is the key OS Login manages and is correct. host_key_checking defaults to false, and it should not: a tunnel to the wrong instance is precisely what host key checking catches.

On the daemon side, validate before restarting. A broken sshd_config on a host reachable only through IAP, with no console, is an outage:

- name: Enforce sshd configuration
  ansible.builtin.template:
    src: sshd_config.j2
    dest: /etc/ssh/sshd_config.d/60-hardening.conf
    mode: "0600"
    validate: /usr/sbin/sshd -t -f %s
  notify: Restart sshd

validate runs the daemon's own parser against the candidate file and refuses to install it if it fails. It is one line and it is the difference between a failed task and a lost fleet.

Pitfall. IAP TCP forwarding needs the firewall to permit 35.235.240.0/20 to port 22 (§5.14), and that rule is Terraform's to create. A playbook that hangs on connection with no error is usually this, and it looks like an Ansible problem.

28.12 OS Hardening §

§32.1 owns the target state — which settings a hardened Compute Engine instance has and why. This section owns how a benchmark is expressed as a role and, more importantly, how the role proves it worked.

A hardening role has three kinds of task and they are not equally trustworthy:

Prefer a whole managed file to line edits. lineinfile against a config file is a regular expression racing a package upgrade, and the failure mode is a file that matches the regex and means something different. A template owns the file completely and its content is reviewable in the repository.

Use a drop-in directory where the daemon supports one — /etc/ssh/sshd_config.d/, /etc/sysctl.d/ — so the role's file is additive and a package upgrade does not fight it.

The assertion is what makes it evidence. Converging a setting and reading the setting back are different claims, and only the second survives an auditor's question (§29.11):

- name: Collect sshd effective configuration
  ansible.builtin.command:
    cmd: /usr/sbin/sshd -T
  register: os_baseline_sshd_effective
  changed_when: false
  check_mode: false

- name: Assert password authentication is disabled
  ansible.builtin.assert:
    that:
      - "'passwordauthentication no' in os_baseline_sshd_effective.stdout | lower"
    fail_msg: "sshd still accepts password authentication on {{ inventory_hostname }}"

The complete role is in code/ansible/roles/os-baseline/.

changed_when: false on a read-only command stops it reporting a change on every run, which is how change output stays meaningful (§28.19).

Pitfall. A hardening role that runs once at build time and never again converges an image, not a fleet. Configuration drifts — an operator edits a file during an incident, a package upgrade replaces one. The role has to run on a schedule against running hosts, and its assertions are what turn that into detection (§28.21).

28.13 Application Deployment §

Chapter 25 owns deployment for containerized workloads and Cloud Deploy. Ansible's remaining deployment role is the VM-based application that does not fit that model.

Prefer an immutable path even here. The strongest pattern on Compute Engine is a golden image built in a pipeline (§8.33, Chapter 23) and rolled out by a managed instance group (§8.11), which gives an artifact that was scanned, a provenance record (§23.13), and a rollback that is a template change.

Where Ansible deploys, it should deploy an artifact, not a build. Fetch a versioned, digest-identified package from Artifact Registry (Chapter 24), place it, and restart the service. A playbook that compiles from source on the host has no artifact to verify (§37.4) and no provenance at all (§37.5).

Deploy serially and check between batches. serial: 2 with a health check task between batches is a rolling deployment that stops rather than continuing into an outage. This is one of the things Ansible genuinely does better than the alternatives on VMs.

The deploy identity is not the configuration identity. §25.13 fixes the separation for pipelines and the same logic applies: an account that can converge OS state should not also be the account that can publish artifacts.

Pitfall. A playbook that both deploys the application and configures the host couples two very different cadences. The host baseline changes quarterly; the application changes daily. Combined, the daily change re-runs the whole baseline and any failure in it blocks a release.

28.14 Configuration Templates §

ansible.builtin.template renders a Jinja2 file to a host. It is where most of the interesting configuration lives and where two specific mistakes recur.

Own the whole file and set its mode explicitly:

- name: Render application configuration
  ansible.builtin.template:
    src: app.conf.j2
    dest: /etc/app/app.conf
    owner: root
    group: app
    mode: "0640"
    validate: /usr/bin/app --check-config %s
  notify: Restart app

mode must be quoted. Unquoted 0640 is parsed as a YAML integer and applied as a different permission set than intended. This is a genuine and common footgun, and it silently produces a world-readable config file.

validate is available on template and copy and should be used wherever the target program can check its own configuration. It is the same argument as §28.11's: a syntactically invalid file installed and then loaded is an outage, and a validated file is a failed task.

Templates are where secrets end up if you are careless. A rendered config containing a password means the plaintext is on the host's disk, which may be correct — the application has to read it somehow — but it must not also be in the repository. §28.16 is the supply route.

Pitfall. A template that renders a secret and a --diff run that prints the rendered file puts the secret in the play output and in the CI log. Mark the task no_log: true when it renders sensitive content, and accept that you have traded debuggability for confidentiality.

28.15 Secrets with Ansible Vault §

Ansible Vault encrypts files and variables at rest in the repository with a symmetric passphrase. It solves a real problem and it has one structural weakness.

What it does well: ansible-vault encrypt_string puts a single encrypted value inline in a group_vars file, so a variable file can be mostly readable with only the sensitive values opaque. --vault-id supports multiple passphrases, so production and development secrets can be separated by who holds which passphrase.

The weakness is the passphrase itself. Vault moves the secret's protection to a key that has to be available wherever a playbook runs — a CI runner, a controller, a laptop. There is no rotation mechanism beyond ansible-vault rekey, no per-secret access control, no audit trail of reads, and no expiry.

The passphrase can come from a script, which is the bridge to §28.16: --vault-password-file accepts an executable, and that executable can fetch the passphrase from Secret Manager. That is a real improvement — it removes the passphrase from disk — and it is still one passphrase protecting everything.

Where Vault is the right tool is content that must live in the repository and is not a GCP-retrievable secret: a licence key, a config file that is sensitive as a whole, an on-premises credential during a migration.

Where it is not is anything Secret Manager can hold, which in this estate is nearly everything (§13.2).

Pitfall. A Vault-encrypted file in Git is encrypted forever at that version. Rotating a secret does not remove the old ciphertext from history, so a later passphrase compromise discloses every historical value. §33.15 owns why history is not the fix.

28.16 Secret Manager Integration §

The google.cloud.gcp_secret_manager lookup plugin reads a secret at run time. It is the answer to §28.15's structural weakness rather than an improvement on it.

The secret is never at rest in the repository, on the controller, or in the inventory — it is fetched during the play, from the service that owns it, with the controller's own keyless identity (§28.6):

- name: Render application configuration
  ansible.builtin.template:
    src: app.conf.j2
    dest: /etc/app/app.conf
    mode: "0640"
  vars:
    db_password: "{{ lookup('google.cloud.gcp_secret_manager',
                            key='billing-prod-db-password',
                            project='rc-saas-shared-sec-01',
                            auth_kind='machineaccount') }}"
  no_log: true

The lookup takes named arguments, not a positional term. Every documented example uses key=, and writing the secret name as a bare positional is the first thing that fails.

Everything §13.x establishes applies unchanged: versions, rotation, replication, and IAM on the individual secret. The controller's identity needs roles/secretmanager.secretAccessor on the specific secret and nothing broader — which is a per-secret access control Vault cannot express, and an access that appears in Data Access audit logs, which Vault reads never do.

A module also exists and creates secrets. Its return_value defaults to true, meaning the secret's plaintext is returned into play results by default — use no_log: true or, better, let Terraform create secrets (§13.13) and let Ansible only read them.

Pitfall. A lookup is evaluated on the controller, not on the target host, and its result is templated into whatever consumes it. Without no_log: true the value appears in verbose output, in the fact cache, and in the CI log — which is the same failure as §28.14's, arriving by a different route.

28.17 Ansible Roles §

A role is Ansible's reusable unit, and the design questions are the same ones §26.7 asks of a Terraform module.

The structure is fixed and worth respecting: tasks/, handlers/, templates/, files/, defaults/, vars/, and meta/. The distinction that matters is defaults/ versus vars/: defaults has the lowest precedence and is the role's input contract; vars has high precedence and is effectively internal. Put anything a caller should set in defaults.

Prefix every variable with the role name. os_baseline_packages, not packages. Ansible has one flat variable namespace per host, so two roles using packages will collide, and the failure is a role silently receiving another role's value.

A role should converge one concern. os-baseline hardens the operating system; app-runtime installs and configures the application. The test is whether the two change on the same schedule — if they do not, they are two roles (§28.13).

Handlers exist so that many changes cause one restart. A role that changes four config files and notifies one handler restarts the service once, at the end. A role that restarts inline restarts it four times, and on a fleet that is an availability event.

Pitfall. Handlers do not run if the play fails before the handler flush. A role that changes a config file and then fails leaves the host with new configuration and a running process using the old one — which is a divergence no subsequent run will notice, because the file is already correct. force_handlers or an explicit meta: flush_handlers is the mitigation.

28.18 Ansible Collections §

A collection is a distributable bundle of modules, plugins, and roles. It is also a supply chain, and this book treats it as one.

Pin collections in requirements.yml with explicit versions, and install from that file in CI. An unpinned collection means the code that configures production changed without a change to the repository — §26.26's argument, applied to a different artifact.

Know what the google.cloud collection actually is. Version 1.14.0 ships 194 modules, one inventory plugin, two lookup plugins, one connection plugin, two filter plugins, and three roles. Every module carries status: preview and supported_by: community. It is generated code with community support, not a Google-supported product.

Google publishes no current documentation for it, which is worth stating plainly: ansible-doc and the collection's source are the authoritative surface, and this chapter was written against them rather than against a documentation page.

Vendor collections for restricted environments. ansible-galaxy collection download produces a tarball set that can be installed offline, which pairs with §27.12's provider mirroring argument: the pipeline should not reach a public registry at run time.

Verify what you install. A collection is Python that runs on the controller with the controller's credentials. It deserves the same treatment as a Terraform provider (§26.2) — a pinned version, a recorded source, and a review when it changes.

Pitfall. ansible-galaxy collection install upgrades transitive dependencies unless told otherwise, so a pinned top-level collection can still bring in a changed dependency. Install from a fully resolved requirements.yml and commit it.

28.19 Idempotency §

Idempotency in Ansible is a property of each module, not of the framework, and the reporting it produces is only as truthful as the tasks.

A module reports changed when it made a change, and that signal is the closest thing Ansible has to drift detection: a converged fleet reports zero changes, and a run that reports changes on a host nobody touched is telling you something.

Which makes false changed reports actively harmful. A command task reports changed every run unless told otherwise, and a role full of them trains everyone to ignore the summary:

- name: Collect effective configuration
  ansible.builtin.command:
    cmd: /usr/sbin/sshd -T
  changed_when: false
  check_mode: false

--check --diff works properly on the configuration side. The core modules this chapter uses — template, copy, lineinfile, user, package, systemd_service — all support check mode with full support and diff mode, so a check run against a fleet genuinely reports what would change.

It does not work on the provisioning side, and that is the chapter's opening argument. No google.cloud resource module supports check mode, so those tasks are skipped rather than simulated. A --check run over a play containing both reports honestly on half of it and silently on the other half, which is worse than reporting on neither.

ansible-lint is the usual static check and it is not installed in this environment, so this book names it as a pipeline stage without prescribing flags or rules.

Pitfall. changed_when: false on a task that does change something hides a real change from the summary and from any alerting built on it. It belongs on read-only commands, and using it to quiet a noisy task is §26.20's ignore_changes mistake in Ansible form.

28.20 Ansible in CI/CD §

An Ansible pipeline is shaped by the fact that it has no plan artifact to approve.

What can be gated before merge:

The check run against a non-production fleet is the closest analogue to a plan, and it is worth building the pipeline around: converge staging, run --check --diff against production, and put that diff in front of a human before the production run.

Authenticate the runner without a key. auth_kind: application with a Workload Identity Federation credential configuration (§28.6, §4.7). The runner's identity should hold read access for the inventory and the specific escalation the playbook needs, and nothing else.

Run with --limit and never with an implicit all. A pipeline that targets a group by name fails visibly when the group is empty; one that targets everything succeeds against a fleet the author did not intend.

Record the run. Ansible's own output is the only record that a convergence happened, so the CI job's log is the evidence artifact (§29.11) and needs the retention that implies.

Pitfall. A pipeline that runs a playbook on every merge to the default branch applies configuration changes with no approval step, because there is nothing to approve. Add the human gate explicitly — an environment approval on the production job — since the tool will not supply one.

28.21 Ansible for Day-Two Operations §

Day-two work — the targeted, urgent, one-off task — is where Ansible is at its least replaceable, and where its lack of a plan matters least.

The ad-hoc form is the point:

ansible gcp_web --limit 'rc-saas-prod-app-*' \
  -m ansible.builtin.systemd_service \
  -a 'name=app state=restarted' --become

A one-off across a dynamic group is exactly the task no other tool in this book does well. Terraform has nothing to say about restarting a service, and doing it by hand across thirty instances is how instances end up in different states.

Use it for detection, not just for change. A read-only playbook of assertions (§28.12) run on a schedule against the fleet reports which hosts have drifted from the baseline, which is the in-guest equivalent of §26.22's drift detection and covers ground Cloud Asset Inventory (§29.6) cannot see.

Every action is attributable. The collection's Google-Ansible-MM-* user agent identifies its API calls in audit logs, and the SSH session itself is attributable through OS Login (§8.19) — so an ad-hoc run is not an anonymous change.

Constrain the blast radius by habit. --limit first, --check before --diff before the real run, and serial for anything that restarts a service. These are conventions rather than controls, which is why they belong in a runbook.

Pitfall. An ad-hoc change is a change nobody reviewed and nothing recorded except a shell history. It is the right tool during an incident and the wrong one afterward: whatever the ad-hoc command fixed must go into a role, or the next rebuild silently reverts it.

28.22 Terraform/OpenTofu and Ansible Together §

This is the seam the whole Part points at, and it works because the two tools meet at a boundary neither has to negotiate: Terraform owns whether a resource exists and what shape it has; Ansible owns what is true inside it.

The handoff is a label, and the API is the interface. Terraform stamps labels on the instance; the dynamic inventory reads them and turns them into groups:

keyed_groups:
  - prefix: gcp
    key: labels.system
  - prefix: gcp
    key: labels.env

There is no shared state file, no generated inventory artifact, and nothing to keep in sync. Terraform writes to the API, Ansible reads from the API, and neither knows the other exists. That is what makes the seam robust — a generated inventory committed to a repository is a copy that goes stale, and every integration built that way eventually does.

The estate's five mandatory labels are already there. B01 requires env, owner, cost-center, data-class, and system on every project, and stamping the same set on instances makes the handoff declarative on both sides for free.

The fallback, for values that are not resource attributes, is reading terraform output -json with ansible.builtin.command and from_json — remembering that each output is wrapped as {"value": …} and the .value is easy to forget. Prefer the inventory: reading Terraform output couples Ansible to the working directory and to state access, and state read access is a privilege the configuration-management identity should not hold (§26.17).

Note that cloud.terraform is a separate collection and is not installed here, so its inventory plugin and lookup are not part of this book's toolchain.

Which tool owns what, decided by capability rather than preference:

The transport closes the loop. Ansible reaches instances that have no external IP through the google.cloud.iap connection plugin (§28.11), authenticating with OS Login identities (§8.19) over IAP TCP forwarding (§8.21) — all three of which are Terraform's to configure. The two tools cooperate without integrating.

Pitfall. The seam breaks when either tool crosses it. Terraform running a local-exec that configures a host creates in-guest state no playbook will reconcile (§26.35); Ansible creating GCP resources creates infrastructure no plan will ever show (§28.2). Both failures are invisible to the other tool, which is exactly why they persist.

Chapter Summary §

• No google.cloud resource module supports check mode — all 79 that declare it are *_info modules — so ansible-playbook --check skips provisioning tasks entirely. Ansible has no terraform plan for GCP resources, and that is why this book provisions with Terraform.
• Core configuration modules do support check and diff fully, so --check --diff is genuinely useful on the half Ansible owns.
• The collection has no IAM binding module, no firewall policy module, no organization policy module, and no CMEK argument anywhere — only customer-supplied keys.
• Every module carries status: preview and supported_by: community, and Google publishes no current documentation; ansible-doc and the source are the authoritative surface.
• The gcp_compute inventory file must end in gcp.yml, gcp.yaml, gcp_compute.yml, or gcp_compute.yaml; any other name is silently ignored and yields an empty inventory.
• hostnames defaults to [public_ip, private_ip, name]; in an estate with no external IPs that default produces unreachable hosts.
• filters uses the Compute Engine aggregatedList language and ANDs multiple entries; omitting zones lists every zone in the project.
• auth_kind has four values and three are keyless: machineaccount (the VM's attached identity), application (ADC, and therefore Workload Identity Federation), and accesstoken. Only serviceaccount consumes a key file.
• The collection's README recommends service account keys; Google's own guidance and this book do not. §28.6 is the default.
• service_account_email works only with machineaccount and selects an attached identity — it is not a way to impersonate.
• google.cloud.iap is a first-party connection plugin that tunnels through IAP; its host_key_checking default of false should be changed.
• Always use validate: on template and copy for daemon configuration — a broken sshd_config on an IAP-only host is an outage.
• Quote mode: in file modules; unquoted 0640 is parsed as an integer.
• Ansible Vault's weakness is the passphrase: no rotation, no per-secret access control, no read audit trail. The Secret Manager lookup removes all three.
• The lookup takes key= as a named argument, not a positional term, and needs no_log: true.
• Prefix role variables with the role name; Ansible has one flat variable namespace per host.
• Terraform owns resource existence, Ansible owns in-guest state, and the handoff is a label the dynamic inventory reads — no shared file, nothing to keep in sync.

Security Checklist §

Sources §

• https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys — Google's guidance against downloaded service account keys (last validated 2026-09-04)
• https://cloud.google.com/iam/docs/workload-identity-federation — federation as the keyless path an ADC credential configuration file consumes (last validated 2026-09-04)
• https://cloud.google.com/compute/docs/reference/rest/v1/instances/aggregatedList — the filter language the inventory plugin's filters option uses (last validated 2026-09-04)
• https://cloud.google.com/iap/docs/using-tcp-forwarding — IAP TCP forwarding, which the google.cloud.iap connection plugin tunnels through (last validated 2026-09-04)
• https://github.com/ansible-collections/google.cloud — the collection's README, its service-account-key recommendation, and its source (last validated 2026-09-04)