Infrastructure as Code Hardening

This chapter sits one layer below the Terraform-and-policy-as-code territory of IaC security & policy as code: once a virtual machine, a bare-metal host, or a golden image actually exists, something still has to lock down the operating system running on it — disable root SSH login, tighten file permissions, set kernel parameters, enforce password aging — against a published rulebook like the CIS Benchmarks. That something is a configuration-management tool, and across the CDP toolchain it's almost always Ansible. This chapter covers how a CIS-hardening Ansible role is actually built, what makes its tasks idempotent, and — the part a live practical exam cares about — how a hardening playbook earns the right to touch a real fleet by clearing a CI gate first. Practical DevSecOps doesn't publish a scored breakdown of its five live challenges, so treat the chapter numbering here as this course's own study framework rather than an official domain weighting — confirm the current exam structure on the vendor's own page.

Two layers of "infrastructure as code" security

☺ Like you're 10: One check reviews the blueprint before the house is built. This chapter's check walks through the finished house changing locks and sealing windows.

IaC security & policy as code covers scanning Terraform and CloudFormation for known-bad patterns — a public S3 bucket, a security group open to the whole internet — before terraform apply ever runs. Those checks answer one question: does the resource that's about to be created exist safely? This chapter answers a different question about a resource that already exists: is what's actually running on it locked down? Provisioning-time scanning can approve a perfectly reasonable EC2 instance or Azure VM whose base image still ships a default SSH configuration that permits root login, a kernel with permissive network settings, and no password-expiry policy at all — none of that is visible in a Terraform plan, because none of it is a cloud-provider resource. It's operating-system configuration, and it's the job of a configuration-management tool, not an IaC scanner, to close that gap.

Ansible, Chef, Puppet, and Salt all do this job; Ansible dominates the CDP-relevant toolchain because it's agentless (SSH in, run tasks, leave — no daemon to maintain on every host) and its push model maps cleanly onto a CI pipeline that already knows how to run commands over SSH. See CDP foundations & the toolchain for where Ansible sits next to the rest of the exam-relevant tool list.

CIS Benchmarks: the rulebook a hardening playbook implements

☺ Like you're 10: A CIS Benchmark is the published rulebook — "lock this door, turn off this valve" — for one specific operating system, and a hardening playbook is just that rulebook translated into code a computer can run.

The Center for Internet Security (CIS) publishes free, community-developed benchmark documents for dozens of operating systems, cloud platforms, and applications — Ubuntu, RHEL, Windows Server, Amazon Linux, Docker, Kubernetes, and more. Each benchmark is organized into numbered sections (initial setup, services, network configuration, logging and auditing, access and authentication, system maintenance) with individual recommendations underneath, and each recommendation carries two independent labels worth knowing cold:

Exact section numbers shift between benchmark versions and OS releases, so don't memorize a control ID from training material and assume it still applies — always check the PDF you're actually implementing against. CIS's own commercial tool, CIS-CAT Pro, scans a host and produces a scored assessment report directly from the benchmark; the open-source route most of this course's tool list favors instead is OpenSCAP paired with the SCAP Security Guide content (now developed as the ComplianceAsCode/content project) — see the OpenSCAP tool page.

Anatomy of a CIS-hardening Ansible role

☺ Like you're 10: A role is a labeled toolbox — one drawer of tasks, one drawer of the values that change per OS, one drawer of "only restart this if something actually changed."

A hardening playbook is almost never a flat list of tasks in one file — it's built as an Ansible role: a standard directory layout (tasks/, handlers/, vars/, defaults/, templates/, meta/) that Ansible knows how to assemble automatically. Three real open-source lineages illustrate the pattern differently: the ansible-lockdown org maintains hand-written, tag-driven CIS roles per OS and benchmark version (RHEL9-CIS, UBUNTU22-CIS, and similar); the ComplianceAsCode/content project auto-generates Ansible remediation tasks directly from the same XCCDF/OVAL definitions OpenSCAP uses to scan, so the rule that checks and the rule that fixes share one source of truth; and the DevSec Hardening Framework's dev-sec/ansible-collection-hardening pairs each hardening role with a matching InSpec profile (dev-sec/linux-baseline, dev-sec/ssh-baseline) built specifically for the CI verify step covered next.

Whichever lineage a role comes from, three conventions repeat: tasks are tagged by level and section (level1, level2, section_5, a specific rule ID) so a caller can run a subset with --tags level1; OS-specific values live in vars/RedHat.yml and vars/Debian.yml, selected at runtime from ansible_facts['os_family']; and a service is only restarted through a handler that fires on change, never unconditionally on every run. A trimmed example:

# roles/cis_hardening/tasks/main.yml
- name: Include OS-family-specific variables
  ansible.builtin.include_vars: "{{ ansible_facts['os_family'] }}.yml"
  tags: always

- name: "5.2.x | Ensure SSH root login is disabled"          # exact rule ID varies by benchmark version
  ansible.builtin.lineinfile:
    path: /etc/ssh/sshd_config
    regexp: '^#?PermitRootLogin'
    line: 'PermitRootLogin no'
    validate: /usr/sbin/sshd -T -f %s
  notify: restart sshd
  tags: [level1, section_5]

- name: "3.3.x | Ensure reverse path filtering is enabled"
  ansible.posix.sysctl:
    name: net.ipv4.conf.all.rp_filter
    value: '1'
    state: present
    sysctl_set: true
    reload: true
  tags: [level1, section_3]

- name: "5.5.x | Ensure password expiration is configured"
  ansible.builtin.lineinfile:
    path: /etc/login.defs
    regexp: '^PASS_MAX_DAYS'
    line: "PASS_MAX_DAYS {{ password_max_days }}"        # from vars/RedHat.yml or vars/Debian.yml
  tags: [level1, section_5]

Nothing in that file is exotic — that's deliberate. A hardening role isn't a research artifact; it's a plain translation of a rulebook into tasks a machine can re-run identically on every host.

Idempotency: the property that makes re-running a playbook safe

☺ Like you're 10: Idempotent means: do it once, do it again, nothing extra happens the second time. It doesn't mean the thing you did was right — just that repeating it is safe.

An Ansible task is idempotent when running it against a system already in the desired state reports "ok" (unchanged) rather than "changed," and never fails, duplicates an effect, or drifts the system further with each run. This isn't automatic — it's a property of the module, and it's why hardening roles are built almost entirely from Ansible's built-in, state-aware modules rather than raw shell commands. ansible.builtin.lineinfile reads the file and only rewrites the matching line if it doesn't already say the right thing; ansible.posix.sysctl reads the live kernel value before deciding whether to set it; ansible.builtin.template renders the file in memory and diffs it against the target before touching disk. Compare that to the anti-pattern it replaces — shell: echo 'PermitRootLogin no' >> /etc/ssh/sshd_config — which appends a duplicate line on every single run and never notices the file is already correct.

Two more habits reinforce this. Ansible's --check mode (paired with --diff) previews what a run would change without applying it, useful for a quick sanity pass — though not every module honors check mode perfectly, so it's a preview, not proof. And handlers exist specifically so a service restart is notified by a change rather than unconditional: a hardening role that restarts sshd on every run, whether or not the config actually changed, isn't idempotent in spirit even if each individual task technically is.

Testing a hardening playbook in CI before anyone trusts it

☺ Like you're 10: Before the playbook is allowed anywhere near real servers, it has to survive a gauntlet: get its grammar checked, run once, run again to prove nothing changed the second time, and then get inspected by a completely separate tool to confirm it actually did the job right.

Molecule is the standard framework for testing Ansible roles in CI, and a hardening role's Molecule scenario walks through a fixed sequence: lint (ansible-lint, yamllint) catches style and known-bad-pattern violations before anything runs; create spins up a disposable target — typically a Docker container built from an image with systemd support, since most hardening roles manage services; converge applies the role to that fresh target; idempotence runs converge a second time and fails the build if any task reports "changed" — this is the automated form of the property from the last section, enforced rather than assumed; and verify runs an independent check against the converged host, using the DevSec Hardening Framework's InSpec profiles, Testinfra, or native Ansible assertions, to confirm the actual CIS controls hold — not just that the role ran without error. Molecule's built-in verifier support has shifted across major versions, so confirm current options against Molecule's own docs before picking one.

# molecule/default/molecule.yml
dependency:
  name: galaxy
driver:
  name: docker
platforms:
  - name: "${MOLECULE_DISTRO:-rhel9-cis}"
    image: "geerlingguy/docker-${MOLECULE_DISTRO:-rockylinux9}-ansible:latest"
    pre_build_image: true
    privileged: true
    cgroupns_mode: host
provisioner:
  name: ansible
verifier:
  name: ansible          # runs verify.yml; pair with a separate InSpec run for the real CIS assertions

# .github/workflows/hardening-role-ci.yml
name: hardening-role-ci
on: [pull_request]
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install ansible-lint yamllint
      - run: yamllint .
      - run: ansible-lint --profile production
  molecule:
    needs: lint
    runs-on: ubuntu-latest
    strategy:
      matrix: { distro: [rockylinux9, ubuntu2204] }
    env:
      MOLECULE_DISTRO: ${{ matrix.distro }}
    steps:
      - uses: actions/checkout@v4
      - run: pip install "molecule-plugins[docker]" ansible
      - run: molecule test        # lint → create → converge → idempotence → verify → destroy

Merges are blocked on any red stage — the pipeline never gets past a failing idempotence run or a failing verify run to reach the merge button. That's the same shift-left discipline covered in security in CI/CD, applied one layer down from application code: the hardening role is the artifact, and it doesn't ship until it clears its own gate.

Keeping the fleet hardened after the first run

☺ Like you're 10: Locking every door once isn't the end of the job — someone can still prop one open later, so the walkthrough has to happen again and again, not just on move-in day.

A role that passes CI and gets applied once has only solved half the problem — the same half IaC security & policy as code solves at deploy time for Terraform. A host hardened on day one can drift: an engineer manually re-enables root SSH login to debug something at 2 a.m. and forgets to revert it, or a package update silently resets a config file this role manages. In practice, fleets stay hardened the same way GitOps keeps a cluster reconciled — by re-running the check on a schedule rather than trusting a one-time apply. That's a scheduled job template in Ansible Automation Platform / AWX, or ansible-pull run from cron on each host so it re-converges against its own copy of the role periodically, correcting drift the same loop caught it the first time. Compliance as code at scale covers how the evidence from those recurring runs — and from a periodic oscap xccdf eval against the same benchmark profile — gets collected and turned into something an auditor can actually consume, rather than re-derived by hand every quarter.

Where this trips people up

☺ Like you're 10: A few ways people fool themselves into trusting a hardening playbook before it's actually earned that trust.

• Idempotent ≠ compliant. A role can converge cleanly and pass its own idempotence check while still failing the CIS control it claims to implement, if a task's logic is simply wrong. Idempotence proves "running it twice does nothing extra," not "it did the right thing once" — that's why Verify is a separate, independent stage from Converge, not a restatement of it.
• Level 2 isn't a better default — it's a different tradeoff. Applying Level 2 controls fleet-wide by default (disabling USB storage, disabling core dumps, aggressive session timeouts) can break real functionality on hosts that never needed that posture. Know which profile a control belongs to before you tag it always.
• The wrong test target gives false confidence. A role that passes molecule test against a generic container image doesn't guarantee identical behavior against your real AMI or golden image — especially for kernel-level controls a container can't fully exercise (see the warning above).
• --check mode is a preview, not a CI gate. Ansible's dry-run mode predicts what would change without applying it, and not every module honors it faithfully. It's useful for a human sanity check; it isn't a substitute for Molecule's idempotence stage, which is a real second run against real converged state.
• Unpinned roles and collections drift silently. A hardening role with no version pin in requirements.yml picks up upstream changes — including new or altered rules — on the next CI run, unreviewed. Pin ansible.posix, community.general, and any third-party CIS role by exact version, the same discipline covered for application dependencies in software composition analysis.