Supply-Chain Security & SBOM

Shift-Left Security for DevOps answers one question well: is the code about to ship free of the vulnerabilities a scanner already knows how to look for? This page answers a different question, one SAST, DAST, and SCA gates were never built to answer: is the artifact that actually reaches production the exact same thing your pipeline built and tested — assembled from ingredients you can name, by a process nobody quietly altered in between? A pipeline can pass every scanning gate cleanly and still ship a backdoor, if the tampering happened after the scan ran, or lives inside a dependency four layers down that nobody enumerated, or was inserted by a maintainer nobody thought to distrust. Software supply-chain security is the discipline built to close that specific gap: knowing exactly what's inside what you ship (the SBOM), proving it wasn't tampered with (signing), proving how it was actually built (provenance), and having a shared vocabulary for how much of that you can honestly claim (SLSA). It became its own discipline for a concrete reason — three real incidents inside four years forced the industry to admit that "the scan was clean" and "nothing was tampered with" had never been the same claim.

Why supply-chain security became its own discipline

☺ Like you're 10: A scanner can only warn you about problems it knows to look for, inside code it can actually see — it says nothing about whether the thing you tested is the thing that got deployed, or whether it was even built the way it claims to have been.

For most of the industry's history, "secure software" meant scanning the code you wrote and the dependencies you declared, then shipping whatever came out clean. That model has a blind spot it was never designed to cover: it assumes the build process itself is trustworthy, that the artifact leaving the pipeline is the same one that entered the registry, and that every dependency in the tree is exactly what its name claims. Three incidents inside four years made that assumption impossible to keep making.

SolarWinds / SUNBURST (December 2020) — attackers didn't exploit a vulnerability in SolarWinds' Orion product code at all; they compromised the build system that produced it, injecting malicious code during compilation so that a legitimately signed, seemingly clean update shipped a backdoor to roughly 18,000 customers, including multiple U.S. federal agencies. Every SAST scan of the source repository would have come back clean, because the source repository was never the thing that was tampered with.

Log4Shell / CVE-2021-44228 (December 2021) — a remote-code-execution flaw in Log4j, a logging library buried as a transitive dependency (a dependency of a dependency, sometimes several layers deep) inside an enormous fraction of the world's Java software. The technical fix was a version bump. The operational nightmare was a question almost nobody could answer quickly: do we even use this, and where? Most organizations had no reliable inventory of their own transitive dependency trees, so "are we affected" took days of manual archaeology instead of the seconds it should have taken.

The xz backdoor / CVE-2024-3094 (March 2024) — someone using the handle "Jia Tan" spent roughly two years building trust as a co-maintainer of xz-utils, a compression library used almost everywhere, before quietly inserting an obfuscated backdoor — not into the visible source, but into the build scripts that assemble the release tarball — that would have granted attackers SSH access on affected systems. It was caught essentially by chance: a Microsoft engineer, Andres Freund, noticed an unexplained ~500ms of SSH login latency while profiling something unrelated, and pulled the thread. A single-character change in trust — one maintainer nobody had reason to doubt — nearly compromised a meaningful slice of the internet's SSH infrastructure.

Governments and standards bodies responded in kind: U.S. Executive Order 14028 (May 2021) directed federal agencies to require an SBOM from software vendors; NIST published the Secure Software Development Framework (SP 800-218); the NTIA defined a minimum baseline of fields an SBOM has to carry to be useful at all; and the Linux Foundation's OpenSSF (Open Source Security Foundation, founded in 2020) became the home for the tooling — Sigstore and SLSA among them — this page is built around. None of that tooling exists to catch a known vulnerability faster. It exists to answer a question shift-left scanning was never built to answer: is this actually the thing we built, unaltered, and can we prove it?

The attack surface: four places to compromise a chain

☺ Like you're 10: An attacker doesn't need to find a bug in your code if they can quietly swap an ingredient, poison the kitchen, or relabel the box after it leaves — the vulnerability scanner never even looks at those spots.

It helps to name the specific gaps this whole discipline is closing, because each one calls for a different countermeasure later in this page. A supply chain has (at minimum) four places trust can be forged, and none of them are "a bug in the code you wrote":

• Source — a hijacked maintainer account, a typosquatted package name (reqeusts instead of requests), or dependency confusion: security researcher Alex Birsan showed in 2021 that publishing a public package with the same name as a company's internal-only package, at a higher version number, made many companies' own build tooling prefer the attacker's public package over their private one — the technique landed him bug bounties from Apple, Microsoft, Tesla, PayPal, and dozens of others.
• Build — the exact SolarWinds pattern: compromise the CI runner or build environment itself so it injects something the source repository never contained. The source can be perfectly clean and the output still backdoored, because nothing checked whether the process was trustworthy.
• Artifact in transit — the binary or image built in CI gets swapped for a different one somewhere between the build job and the registry it's pushed to, before anyone signs or verifies anything.
• Registry / deploy — a mutable tag like :latest gets silently overwritten after the version that passed review, or the deploy step pulls a different digest than the one that was actually tested, because nothing pinned or verified which specific artifact was supposed to run.

The SLSA framework, covered later on this page, organizes essentially this same list into a formal threat model — it's worth knowing the shape here first, because every tool in the rest of this page maps to closing one specific gap on this diagram, not to finding a CVE.

SBOMs: an inventory you can actually query

☺ Like you're 10: An SBOM is a complete ingredient list for a piece of software — not just what you bought at the store, but every ingredient those ingredients were made from too, all the way down.

A Software Bill of Materials (SBOM) is a machine-readable inventory of every component inside a piece of software — direct dependencies and transitive ones, with names, versions, and (ideally) cryptographic hashes and license information for each. The Log4Shell answer to "do we even use this, and where" is exactly what a good SBOM makes instant instead of archaeological: grep every SBOM in your artifact store for log4j-core, get a precise list of affected services in minutes, not days of manually walking build files across every repo in the org.

Two formats, both worth knowing

Two competing-but-largely-interoperable standards dominate: SPDX (Software Package Data Exchange, originated at the Linux Foundation, now an ISO/IEC 5962:2021 standard) grew out of license-compliance tooling and is strong on licensing metadata; CycloneDX (an OWASP project) grew out of application-security tooling and is strong on vulnerability and dependency-graph metadata. Neither is "more correct" — plenty of organizations produce both, since the generators below emit either with the same command. The NTIA's minimum-elements guidance names the fields that make an SBOM actually useful regardless of format: supplier name, component name, version, a unique identifier (a package URL or hash), the dependency relationships between components, the author of the SBOM itself, and a timestamp. An SBOM missing dependency relationships is just a flat parts list — it can't answer "what depends on the vulnerable thing," which is usually the question that actually matters.

Generating one: from source, or from a built artifact

You can generate an SBOM two ways, and they answer subtly different questions. A source-based SBOM reads your lockfiles and manifests directly (package-lock.json, go.sum, requirements.txt) — fast, and exactly what your dependency-resolution tooling intended to include. An artifact-based SBOM scans the actual built container image or binary — slower, but it catches what genuinely ended up in the shipped thing, including a base-image layer's own packages that never appeared in any of your manifests at all. Syft, an open-source Anchore tool, does both from one binary:

# Artifact-based: scan the image that's actually going to ship
syft registry.acme.io/checkout:1.8.2 -o spdx-json > checkout-1.8.2.spdx.json
syft registry.acme.io/checkout:1.8.2 -o cyclonedx-json > checkout-1.8.2.cdx.json

# Source-based: scan a checkout directly, before an image even exists
syft dir:. -o cyclonedx-json > checkout-source.cdx.json

# Flags and subcommands shift between syft releases — check `syft --help`
# against the version your pipeline actually pins before you script around it.

Generating the file is the easy part; the SBOM only earns its keep once something is actually watching it. Push it into a tool like Dependency-Track (also an OWASP project) or your registry's native SBOM store, and it becomes a standing, continuously re-checked inventory — the next Log4Shell-scale CVE gets matched against every SBOM you've ever generated automatically, the moment the CVE is published, instead of triggering a fire drill of manual greps.

Signing artifacts: from long-lived keys to Sigstore's keyless model

☺ Like you're 10: A signature proves the box wasn't opened and swapped after it left the kitchen — and the newest way to sign doesn't even need a key you have to guard for years, because it mints a brand-new one that expires in minutes.

An SBOM tells you what's supposed to be inside an artifact. It says nothing about whether the artifact you're holding is actually the one the SBOM describes, or whether either was swapped in transit — the "artifact swapped before it's signed" gap from the attack-surface diagram above. Signing closes that gap: a cryptographic signature over the artifact's content digest proves, mathematically, that the bytes weren't altered after the signature was created, and by whom (or what) it was created.

Traditional code signing has a well-known failure mode: a long-lived private key sits somewhere — a build server, a hardware token, a password manager — and whoever holds it can sign anything, forever, until someone remembers to rotate it. That's precisely the static-credential problem Secrets & Credential Management covers in general, applied to a signing key specifically: lose control of it, and every artifact it ever signed is now suspect, with no way to tell which signatures were legitimate.

Sigstore, an OpenSSF project, sidesteps the key-custody problem entirely with keyless signing. Instead of a long-lived key, a CI job authenticates with its platform's OIDC identity (GitHub Actions issues one automatically to every workflow run); Fulcio, Sigstore's certificate authority, verifies that OIDC token and issues a certificate — valid for about ten minutes — binding that specific workflow's identity to a brand-new, ephemeral keypair generated just for this one signature. cosign signs the artifact's digest with that ephemeral private key, and the private key is discarded the moment the job ends — there is nothing left to steal, because nothing long-lived was ever created. The signature and certificate are then recorded in Rekor, a public, append-only transparency log (the same Merkle-tree idea behind Certificate Transparency for TLS), so a signature's existence and timing can be independently verified later even if the ten-minute certificate has long since expired.

# Keyless signing — cosign authenticates via the CI platform's OIDC token,
# Fulcio issues a short-lived cert, Rekor logs the signature. No key to manage.
cosign sign registry.acme.io/checkout@sha256:9f2b1a7c4e...

# Attach the SBOM itself as a signed attestation on the same digest
cosign attest --predicate checkout-1.8.2.cdx.json --type cyclonedx \
  registry.acme.io/checkout@sha256:9f2b1a7c4e...

# Verify later — pin the exact identity, not just "a signature exists somewhere"
cosign verify registry.acme.io/checkout@sha256:9f2b1a7c4e... \
  --certificate-identity="https://github.com/acme/checkout/.github/workflows/release.yml@refs/heads/main" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com"

Verification is the step that actually matters — a signature nobody checks is no better than no signature at all — which is why the --certificate-identity and --certificate-oidc-issuer flags above aren't optional decoration. cosign verify without them merely confirms a valid Sigstore signature exists; with them, it confirms the signature came from the specific workflow you trust, on the specific branch you trust, and not from some other repository's release job that also happens to use keyless signing.

Provenance: proving how, not just what

☺ Like you're 10: The SBOM is the ingredient list; provenance is the recipe card — who cooked it, in which kitchen, starting from which ingredients, at what time.

An SBOM answers "what's inside." A signature answers "was it altered after signing." Neither answers a third question that SolarWinds made unavoidable: how was this actually built, and by what? Provenance is a signed, structured statement — a metadata record, not the artifact itself — describing the build: which source commit it came from, which build system produced it, what the build invocation actually was, and what other artifacts (base images, build-time dependencies) went into it. The standard shape for this statement is in-toto attestation: an envelope holding a predicateType (what kind of claim this is) and a predicate (the claim itself), signed the same way an artifact signature is. SBOM and provenance are both, mechanically, just different predicate types living inside the same attestation format — which is why the same signing tooling handles both.

Manually assembling and signing a provenance attestation is real work, which is why the practical path is usually a platform that generates it for you as a build side-effect. GitHub Actions ships this natively:

# .github/workflows/release.yml — one step, native GitHub build provenance
- uses: actions/attest-build-provenance@v1
  with:
    subject-path: 'dist/checkout-server'
# Produces a signed in-toto attestation recording the workflow, the triggering
# commit, and the repository that built this exact file — no separate tooling.

The tighter version of this pattern is a dedicated build platform generating the provenance itself, rather than your own build script self-reporting it — the difference matters enough that it's the exact axis SLSA's build levels are built around, which is where this page goes next.

The SLSA framework and its build levels

☺ Like you're 10: SLSA is a report card for how hard it would be for someone to fake the recipe card — grading not the food, but how trustworthy the kitchen's own paperwork actually is.

SLSA (Supply-chain Levels for Software Artifacts, usually pronounced "salsa") began as an internal Google framework — the same thinking behind Google's own Binary Authorization — and was donated to the OpenSSF as a public specification. Rather than a single pass/fail bar, SLSA defines graduated levels describing how much you can actually trust an artifact's provenance, from "none" to "an independent build platform, not the build script itself, vouches for this and the process can't be tampered with by the build."

Where you sit on this ladder is mostly a property of your CI platform's own architecture, not something you bolt on separately. GitHub-hosted runners generating attestations through actions/attest-build-provenance, or Google Cloud Build's native provenance, both target Build L3-equivalent guarantees out of the box, because the isolation and non-forgeability live in the platform, not in a script you'd have to write and trust yourself.

Wiring all three into a real pipeline

☺ Like you're 10: Three short steps added to a build file you already own — generate the ingredient list, sign the box, staple on the recipe card — and a fourth step at the door that refuses entry to anything missing one.

Concretely, this is an extension of the same build workflow this course has been assembling since CI/CD pipelines and build & artifact management — a few more steps after the image is built, and one more gate before it's allowed to run anywhere that matters.

# .github/workflows/release.yml — SBOM, sign, and attest, chained after a build
jobs:
  release:
    permissions:
      id-token: write     # required for keyless OIDC signing
      contents: read
      attestations: write
    steps:
      - uses: actions/checkout@v4
      - name: Build and push
        run: |
          docker build -t registry.acme.io/checkout:${{ github.sha }} .
          docker push registry.acme.io/checkout:${{ github.sha }}
      - name: Generate SBOM
        run: syft registry.acme.io/checkout:${{ github.sha }} -o cyclonedx-json > sbom.json
      - name: Sign the digest (keyless)
        run: cosign sign --yes registry.acme.io/checkout@${{ steps.build.outputs.digest }}
      - name: Attach the SBOM as a signed attestation
        run: cosign attest --yes --predicate sbom.json --type cyclonedx \
               registry.acme.io/checkout@${{ steps.build.outputs.digest }}
      - name: Native build provenance
        uses: actions/attest-build-provenance@v1
        with:
          subject-name: 'registry.acme.io/checkout'
          subject-digest: ${{ steps.build.outputs.digest }}

Generating and signing everything is only half the pattern — nothing stops an unsigned image from being deployed unless something in the deploy path actually refuses it. That's an admission-control gate, the same enforcement mechanism Compliance as Code & Policy Enforcement covers in general, pointed specifically at signatures instead of resource-configuration rules:

# Kyverno ClusterPolicy — reject any Pod whose image isn't signed by a
# known workflow identity. Illustrative shape; verify current syntax
# against Kyverno's own docs before applying it to a real cluster.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-signed-images
spec:
  validationFailureAction: Enforce
  rules:
    - name: verify-signature
      match:
        resources: { kinds: [Pod] }
      verifyImages:
        - imageReferences: ["registry.acme.io/*"]
          attestors:
            - entries:
                - keyless:
                    subject: "https://github.com/acme/*/.github/workflows/release.yml@refs/heads/main"
                    issuer: "https://token.actions.githubusercontent.com"

With that policy enforced, a Pod referencing an unsigned image — or one signed by a workflow that isn't this exact release job — is rejected at admission, before kubelet ever pulls it. This is the same "blocks merge vs. blocks deploy" distinction Shift-Left Security for DevOps draws for scanning gates, one stage later in the pipeline: SAST and SCA stop a bad change from merging; this gate stops an unverified artifact from running, regardless of how it got that way.

A practical maturity path — and exactly how this differs from shift-left

☺ Like you're 10: Start by writing down every ingredient, then seal the box, then teach the front door to reject any box that isn't sealed — you don't have to do all three on day one.

Adopting everything on this page at once is a good way to adopt none of it well. The order that actually works, roughly in increasing effort:

Steps 1–2 are genuinely a day's work bolted onto a pipeline you already own. Step 3 is where most teams stall, for the same reason a too-strict scanning gate gets quietly disabled: turning on enforcement before every legitimate image in the org is actually signed just breaks deploys, and a broken deploy gate gets bypassed within a week. Roll enforcement out namespace by namespace, the same staged-rollout discipline Feature Flags & Progressive Delivery applies to application changes, applied here to a policy change instead.

This closes the loop Secrets & Credential Management opened when it promised that signing keys are, themselves, secrets to manage — keyless signing is the specific design choice that removes that problem for the common case, by never creating a long-lived key in the first place. And it closes the loop Shift-Left Security for DevOps left open at its own end: the provenance and SBOM side of "prove exactly what shipped," rather than "scan it before it does." For the deeper specialist treatment — container base-image hardening, minimal images, and the full SLSA threat model applied to a real threat-modeling exercise — see the DevSecOps course's Container & Supply-Chain Security.