Compliance as Code at Scale

Chapter 7 built the Ansible role that hardens one host and gave that role a CI gate of its own — lint, converge, prove idempotence, verify. This chapter answers the question that gate leaves wide open: how do you prove — continuously, and across every host actually running in production, not just the one you happened to test in Molecule — that the controls still hold, in a form an auditor will accept instead of a screenshot someone took in a hurry? Two tools carry this chapter, and both turn a written rule into something a machine can execute and re-run: Chef InSpec, which expresses a control as a testable resource in a small Ruby DSL, and OpenSCAP, which runs the same kind of check against the machine-readable XCCDF and OVAL content the SCAP Security Guide publishes. Both produce evidence. This chapter is about writing that evidence once, running it against a whole fleet instead of a single canary host, and wiring the aggregated result into a pipeline gate — not about which regulatory framework it happens to satisfy, which is a different chapter's job entirely.

Chapter 8 of 9: from one hardened host to evidence across a fleet

☺ Like you're 10: This isn't the "which framework do I need" page — that's already been covered. This is the "how does the proof actually get generated and collected" page.

A quick honesty note before anything else, same as every chapter in this blueprint: Practical DevSecOps doesn't publish a scored breakdown of its five live challenges, so treat the chapter numbering in this Exam Blueprint section as this course's own study framework rather than an official domain weighting — confirm the current exam structure on the vendor's own page. Within that framework, this chapter is deliberately narrow and mechanical. It does not re-cover the five frameworks (SOC 2, ISO 27001, PCI-DSS, HIPAA, GDPR), the point-in-time-vs-continuous distinction, or how one control maps to several frameworks at once — that's compliance & governance's job, and it does it well. It also does not re-cover how a hardening role actually fixes a host, or that role's own Molecule CI gate — that's infrastructure as code hardening, chapter 7, and this chapter picks up exactly where that one left off. What's left, and what this chapter is entirely about, is four mechanical steps: (1) write a control as a testable InSpec resource, (2) run the equivalent check with OpenSCAP against published XCCDF/OVAL content, (3) run both of those across a fleet of many hosts rather than one, and (4) turn the resulting pile of per-host results into one aggregated evidence stream that gates a pipeline and satisfies an auditor at the same time.

Scope it against its neighbors precisely: chapter 7 answers "is this host's configuration correct" with a tool that fixes things (Ansible). This chapter answers "can I prove, for every host, that it's correct" with tools that only check and report — InSpec and OpenSCAP never modify a system by default. And chapter 9, vulnerability management & triage, picks up after this chapter ends — once you have a queue of failed controls across a fleet, that's where you learn how to prioritize and work through it.

InSpec: turning a written control into a testable resource

☺ Like you're 10: Instead of writing "root login should be off" in a document nobody re-reads, you write it as a tiny program that actually goes and checks — and tells you true or false, every single time you run it.

Chef InSpec is an open-source framework for writing infrastructure and compliance tests as code. A test is called a control, controls live inside a profile (a directory with a bit of required structure), and each control wraps a describe block around a resource — a built-in object InSpec knows how to inspect on a target system — and asserts something about it with a matcher. Take the exact rule chapter 7's Ansible task implemented — "Ensure SSH root login is disabled" — and write it as an InSpec control instead of a fix:

# controls/sshd.rb
control 'sshd-01' do
  impact 1.0
  title 'Disable SSH root login'
  desc 'Root must not be permitted to authenticate directly over SSH.'
  tag cis: 'CIS Ubuntu 22.04 LTS Benchmark v1.0.0 5.2.10'   # exact section ID varies by benchmark version
  tag severity: 'high'
  ref 'CIS Ubuntu 22.04 LTS Benchmark', url: 'https://www.cisecurity.org/benchmark/ubuntu_linux'

  describe sshd_config do
    its('PermitRootLogin') { should cmp 'no' }
  end
end

Every piece of that control is deliberate. impact is a float from 0.0 to 1.0 that InSpec's own reporters use to bucket severity (roughly: 0.7–1.0 critical, 0.4–0.7 major, below that minor) — it's what a downstream threshold check reads, not just documentation. tag attaches arbitrary metadata — a CIS section reference, a severity label, an internal ticket ID — that later tooling can filter and group on. sshd_config is a built-in resource: InSpec ships dozens of these (file, package, service, port, command, docker_container, cloud-provider resources like aws_security_group, and many more), each one knowing how to read one specific kind of system state. cmp is a matcher built for exactly this kind of comparison — it's more forgiving than a strict string match, coercing types sensibly so should cmp 'no' matches whether the underlying value came back as a string, a symbol, or mixed case.

A profile is more than one file — it's a small package with its own metadata, and profiles can depend on other profiles the way a Terraform module depends on another module:

# inspec.yml
name: acme-linux-baseline
title: Acme Linux Baseline
maintainer: Acme Platform Security
license: Apache-2.0
summary: Acme's CIS-aligned baseline, layered on the DevSec community baseline.
version: 2.3.0
depends:
  - name: linux-baseline
    git: https://github.com/dev-sec/linux-baseline
    tag: v2.11.0
  - name: ssh-baseline
    git: https://github.com/dev-sec/ssh-baseline
    tag: v2.8.0
supports:
  - platform: linux

That depends block is the same composition pattern chapter 7 described for hardening roles — the DevSec Hardening Framework ships a matching InSpec profile for nearly every one of its Ansible collections (dev-sec/linux-baseline, dev-sec/ssh-baseline, dev-sec/apache-baseline, dev-sec/mysql-baseline, and more), so an organization rarely writes CIS controls from scratch — it inherits the community baseline and layers on a handful of org-specific controls like sshd-01 above. Running it against a real target looks like this:

inspec exec . \
  --target ssh://ops@10.0.4.21 \
  -i ~/.ssh/fleet_ed25519 \
  --reporter cli json:results/web-01.json

Profile: Acme Linux Baseline (acme-linux-baseline)
Version: 2.3.0
Target:  ssh://ops@10.0.4.21

  ✔  sshd-01: Disable SSH root login
     ✔  SSH Configuration PermitRootLogin is expected to cmp == "no"
  ✔  os-05: Ensure /etc/shadow permissions are 0000
  ×  sshd-02: Disable password authentication
     ×  SSH Configuration PasswordAuthentication is expected to cmp == "no"
        expected: "no"
        actual:   "yes"

Profile Summary: 41 successful controls, 1 control failure, 0 controls skipped
Test Summary:    52 successful, 1 failure, 0 skipped

The --reporter cli json:results/web-01.json flag is the important part for everything later in this chapter: InSpec can emit a human-readable summary and a structured JSON file in the same run, and the JSON file — not the terminal output — is the actual evidence artifact that flows into aggregation.

OpenSCAP: XCCDF checklists and OVAL definitions at machine scale

☺ Like you're 10: XCCDF is the checklist's table of contents — which rules belong to which profile. OVAL is the actual step-by-step instructions each rule follows to check the real machine. OpenSCAP is the program that reads both and runs the check.

SCAP (the Security Content Automation Protocol) is a NIST-maintained suite of open standards for expressing security checks in a machine-readable, vendor-neutral way, and OpenSCAP is the open-source toolset — command-line utility oscap plus supporting libraries — that implements it. Two component standards inside SCAP matter most for this chapter, and confusing them is a genuine, common exam trap:

Neither is useful alone for this exam's purposes — an XCCDF rule with no OVAL definition behind it is a title with no test, and a pile of OVAL definitions with no XCCDF wrapping them is untitled, unselectable logic. The SCAP Security Guide project (developed today as ComplianceAsCode/content — the same project chapter 7 named as the source of auto-generated Ansible remediation) publishes both bundled together as a single datastream file per operating system, along with CPE (platform identification) data so a scan knows which rules even apply to the host in front of it:

oscap xccdf eval \
  --profile xccdf_org.ssgproject.content_profile_cis_level1_server \
  --results-arf arf-$(hostname).xml \
  --report report-$(hostname).html \
  /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

Exact profile IDs and datastream file names shift with benchmark and OS version — run oscap info /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml against the real file on a real host to list the profiles it actually contains before assuming an ID from training material still applies. --results-arf is worth calling out specifically: it produces an ARF (Asset Reporting Format) file, which bundles the target system's characteristics, the XCCDF results, and the underlying OVAL results into one interchange-ready document — the OpenSCAP equivalent of InSpec's JSON reporter output, and the artifact this chapter's aggregation step actually consumes.

Two more tools in the OpenSCAP family matter specifically for scanning at scale rather than one host at a time: oscap-vm scans a VM disk image (qcow2, raw) offline via libguestfs, without booting it, and oscap-podman scans a container image the same way — both let you scan the golden image once, before it's ever instantiated, instead of scanning every host cut from it individually. That distinction matters for the fleet math in the next section: an image-scan result and a fleet-scan result answer genuinely different questions.

Evidence at fleet scale: one profile, every host, one dashboard

☺ Like you're 10: One host's clean checklist only tells you about that one desk. "At scale" means running the same checklist on every desk in the building and keeping every single completed sheet somewhere searchable — not stacking them in a drawer nobody opens again.

Everything so far runs against one target. A real fleet is dozens, hundreds, or thousands of hosts, running a mix of InSpec profiles and OpenSCAP scans depending on the host's role and the team that owns it — and the two tools don't emit the same file format. MITRE's Security Automation Framework (SAF) CLI exists specifically to close that gap: it normalizes both InSpec JSON and OpenSCAP's ARF/XML into one common schema, the Heimdall Data Format (HDF), so a fleet with mixed tooling still produces one shape of evidence to aggregate, score, and view.

Two commands do the actual work in that middle of the diagram:

# normalize each host's evidence, whichever tool produced it, into one schema
saf convert inspec2hdf -i results/web-01.json -o hdf/web-01.hdf.json
saf convert oscap_xml2hdf -i arf-db-02.xml -o hdf/db-02.hdf.json

# roll every HDF file in the fleet into one aggregate score, checked against a written bar
saf validate threshold -i "hdf/*.hdf.json" -F threshold.yml

A threshold file is a small, deliberate policy document, not a magic number InSpec or OpenSCAP invents on your behalf — you write the bar:

# threshold.yml
compliance:
  min: 90               # aggregate pass percentage across the whole fleet, not per host
results:
  failed:
    critical:
      max: 0             # zero tolerance — any critical failure anywhere in the fleet fails the gate
    high:
      max: 0

SAF's exact threshold schema has evolved across releases, so confirm current field names against the tool's own docs before wiring one into a gate — the shape that matters for this exam is the idea itself: a minimum aggregate compliance percentage, plus a hard zero-tolerance ceiling on the severities you've decided can never ship, checked in one command against the whole fleet's normalized evidence at once. The scored, browsable version of the same data lives in Heimdall2 (or the client-side-only Heimdall Lite, which never uploads evidence anywhere — useful when the evidence itself is sensitive) — MITRE's open-source viewer for HDF files, giving a security or compliance team one dashboard that drills from "236 of 240 hosts compliant" down to the exact failing control on the exact host, regardless of which of the two scanning tools produced that particular result.

Wiring the evidence into a pipeline gate

☺ Like you're 10: The last step is making the computer actually stop and say no when the fleet doesn't pass — and keeping every checklist afterward, not just the ones that failed.

Putting the whole chapter together, a fleet-wide compliance stage loops the scan across an inventory, normalizes every result, and gates on the aggregate — failing the build the same way any other required check does:

# .ci/pipeline.yml — fleet compliance evidence, generated and gated in one stage
compliance-evidence:
  stage: compliance
  script:
    - mkdir -p evidence hdf
    - ansible-inventory --list -i inventory/prod.yml | jq -r '._meta.hostvars | keys[]' > hosts.txt
    - |
      while read -r host; do
        inspec exec acme-linux-baseline \
          --target "ssh://ops@${host}" -i "$SSH_KEY" \
          --reporter "json:evidence/${host}.json" || true
      done < hosts.txt
    - saf convert inspec2hdf -i "evidence/*.json" -o hdf/
    - saf validate threshold -i "hdf/*.hdf.json" -F threshold.yml   # non-zero exit fails the job
  artifacts:
    paths: [evidence/, hdf/]
    expire_in: 2 years   # audit retention, not a CI convenience window
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_PIPELINE_SOURCE == "schedule"'   # also runs nightly against the whole fleet, not just on a merge

Three details in that pipeline are load-bearing, not incidental. First, the loop runs per-host with || true so one unreachable host doesn't abort the whole scan before the rest of the fleet gets checked — a single dead SSH connection shouldn't hide every other host's result. Second, expire_in: 2 years is a deliberate departure from typical CI artifact retention, which is usually measured in days — this artifact isn't a build cache, it's the audit trail itself, and it needs to survive as long as the audit window that might sample it. Third, the pipeline runs on both a merge request and a schedule — a merge-triggered scan only ever checks the hosts affected by that change; a nightly scheduled run is what actually catches the host that drifted out of compliance three weeks after its last deploy, the same way infrastructure as code hardening covered a role staying correct only if it keeps re-running, not because it ran correctly once.

Waivers and tailoring without losing the paper trail

☺ Like you're 10: Sometimes a rule really shouldn't apply to one specific host — but "I decided to skip it" only stays trustworthy if you also write down why, and by when someone has to check that reason is still true.

Not every failed control should block a merge forever. A bastion host might legitimately need a setting a general-purpose server shouldn't, or a control might simply not apply to a given role. Both InSpec and OpenSCAP support documented exceptions — and both, done right, keep the exact same discipline static analysis & secrets detection established for a SAST suppression: a reason and an expiration date, or it isn't a documented risk decision, it's a vulnerability with extra steps.

InSpec's mechanism is a waiver file, passed at execution time:

# waivers.yml
sshd-01:
  run: false
  justification: "Bastion host gates SSH behind a hardware token proxy; risk accepted, ticket SEC-2026-014."
  expiration_date: 2026-12-31

inspec exec acme-linux-baseline \
  --target ssh://ops@bastion-03 -i "$SSH_KEY" \
  --waiver-file waivers.yml \
  --reporter json:evidence/bastion-03.json

OpenSCAP's equivalent isn't a runtime flag — it's a tailoring file, an XCCDF document that layers on top of the base profile to select or deselect specific rules, typically authored in SCAP Workbench and exported as its own tailoring.xml. The meaningful difference from an ad hoc --skip-rule flag typed on a command line is that a tailoring file is itself a versioned, reviewable artifact — committed to the same control repo as the datastream it modifies, so the exception is exactly as auditable as the profile it changes, not a one-off decision buried in a shell history nobody can reconstruct six months later.

oscap xccdf eval \
  --tailoring-file tailoring.xml \
  --profile xccdf_org.ssgproject.content_profile_cis_level1_server_bastion_customized \
  --results-arf arf-bastion-03.xml \
  ssg-rhel9-ds.xml

Common exam traps

☺ Like you're 10: These are the specific ways someone who understands InSpec and OpenSCAP individually still gets the "at scale" part wrong.

• Mistaking one clean host for fleet compliance. Running inspec exec against a single dev box or the golden image and calling the fleet compliant conflates "this one artifact passes" with "everything cut from it, running in production right now, still matches." Drift after boot is real — see chapter 7's own warning about it — which is exactly why this chapter's pipeline scans the running inventory, not just the image it came from.
• Confusing XCCDF and OVAL. XCCDF is the checklist structure — profiles, groups, which rules belong to which named benchmark. OVAL is the actual testable logic a rule points to. A question about "which standard defines the profile a scan selects" is asking about XCCDF; a question about "which standard defines the test logic itself" is asking about OVAL.
• Running --remediate directly against production. Auto-generated remediation is still a change to a live system, and it deserves chapter 7's converge → idempotence → verify gate before it's trusted at scale — not a live, ungated run the first time a scan turns something red.
• Treating a raw InSpec exit code as the whole answer. inspec exec returns 0 when every control passes and a nonzero code otherwise — historically 100 for at least one failed control and 101 for at least one skipped control with none failed, though exact codes have shifted across major versions, so confirm current behavior before wiring a bare exit code into a gate rather than a proper threshold check against the JSON. A nonzero exit alone also can't distinguish "one low-impact control skipped" from "a critical control failed on forty hosts" — that distinction is exactly what a threshold.yml exists to encode.
• A waiver or tailoring file with no expiration date. Covered at length above because it's the single most common way "we accepted this risk" quietly becomes "we forgot this risk exists" — the same failure mode chapter 5 flags for an unreviewed SAST suppression, just one layer further from the code.

Chapter 9 covers what happens once a fleet-wide scan like this hands you a queue of failed controls to work through: vulnerability management & triage. For the frameworks this evidence actually gets mapped against, see compliance & governance; for the hardening role this chapter's evidence verifies, see infrastructure as code hardening. And for the two tools named in this chapter on their own pages, see the InSpec tool page and the OpenSCAP tool page.