DevOps in Depth · FinOps for Delivery Pipelines

FinOps for Delivery Pipelines

There are two cloud bills in most engineering organizations, and only one of them gets watched. The first is production — the servers, databases, and load balancers that serve customers, and the subject of general cloud FinOps. The second is the delivery pipeline itself — the fleet of CI runners that compiles, tests, scans, and packages every single commit before it ever reaches production. Nobody's a customer of that fleet; it doesn't show up on a product roadmap; and because CI/CD pipelines run constantly, in the background, on every push, its bill quietly compounds while everyone's attention stays on the servers customers actually touch. This page is scoped narrowly and deliberately: not cloud cost management in general, but the specific, recurring spend generated by the act of shipping code — runner minutes, redundant builds, and the shared infrastructure twenty teams push into at once.

☺ Explain it like I'm 10

Picture a school copy room with one rule: any time a single kid fixes one typo on one page of their group project, the office reprints the entire 500-page binder, in full color, on the fanciest laser printer in the building — even though 499 pages didn't change and most of it was never in color to begin with. Do that every time any kid finishes anything, all day, every day, and the copy room's toner bill quietly outgrows the school's actual paper budget. The fix isn't "make fewer corrections" — that just means slower homework. The fix is noticing: reuse the 499 pages that didn't change, only print in color when a page actually needs it, and only reach for the fancy printer when the cheap one won't do. FinOps for delivery pipelines is that same noticing, applied to a build system instead of a copy room.

🦥🦫Your hosts for this topic: Sol the Sloth & Benny the Beaver — Benny built the pipeline and is the one who has to change how it runs; Sol is the one who sits down afterward with the runner-minute bill and does the slow, honest arithmetic on what it actually cost to ship.

Why the pipeline needs its own cost lens

☺ Like you're 10: The production servers aren't the only thing that costs money — the machine that builds and tests your code before it ever reaches them costs money too, and almost nobody's watching that meter.

General cloud FinOps — allocating a shared production bill down to the team and workload that caused it — is its own deep discipline, and this course doesn't repeat it here. What it usually leaves out entirely is the compute layer that runs before production: the CI runners that check out your repo, install dependencies, compile, run every test suite, build container images, and push artifacts, on every single push, every single day, whether or not that push ever ships. That work is real, metered compute — GitHub Actions and GitLab CI both bill it by the minute, self-hosted runners cost real EC2 or on-prem hardware whether they're busy or idle, and CircleCI bills it in credits — and at any organization running more than a handful of repos, it adds up to a line item finance eventually asks about, usually right after someone notices it grew 40% quarter over quarter with no corresponding growth in headcount.

This maps directly onto CALMS's Measurement pillar, the same way the DORA metrics do — you cannot optimize a cost you cannot see — and onto Lean, because small batch sizes were never free; they trade a large, infrequent cost for many small, frequent ones, and if nobody prices the frequent ones, "small batches" quietly becomes "more total spend" instead of the throughput win it's supposed to be. The rest of this page works through four concrete levers — right-sizing, caching, per-team visibility, and pipeline design — and then the one place they all collide: what happens to your CI bill when you chase deployment frequency without touching any of them.

Anatomy of a CI/CD bill

☺ Like you're 10: One push to your code can quietly turn into a dozen separate jobs running at once — and the bill is every one of those jobs' minutes added together, not just the one you were thinking about.

The formula behind every CI invoice is short: cost = rate per minute × minutes run × number of parallel jobs. All three levers matter, but the third is the one teams most often forget to look at, because a pipeline-as-code matrix build is designed to hide it. A workflow that tests against 3 operating systems and 4 language versions isn't one job — it's twelve, all triggered by the same single push, each billed independently. A 5-minute test suite that "feels like" 5 minutes of CI cost is actually 60 runner-minutes the moment a matrix multiplies it, and the person who wrote strategy: matrix for good reasons — real cross-platform coverage — rarely goes back to ask whether every cell in that matrix still earns its keep.

git push 1 commit, trunk 🦥 matrix: ×12 ubuntu · node18 — 6 min, $0.05 ubuntu · node16 — 6 min, $0.05 windows · node18 — 7 min, $0.09 (2×) windows · node16 — 7 min, $0.09 (2×) macOS · node18 — 6 min, $0.48 (10×) macOS · node16 — $0.48 (10×) no fix from this cell in 6mo — pure waste 12 jobs from 1 push · ≈ $1.10/run before caching or right-sizing

Hosted-runner rates are a real, published starting point, not a rumor: GitHub Actions bills Linux 2-core runners per minute, and applies roughly a 2× multiplier for Windows and a ~10× multiplier for macOS on the same job — a rate difference driven by Apple's hardware licensing, not by macOS jobs being ten times more expensive to run. GitLab CI/CD meters usage as a monthly "compute minutes" quota rather than a flat per-minute price, and CircleCI bills in credits that vary by executor class (Docker, machine, macOS, GPU). Treat every number above as illustrative — all three vendors revise pricing and free-tier allowances often enough that you should read them straight off the vendor's current pricing page before building a budget on them, the same hedge this course's own certification pages give exam costs and formats.

Right-sizing CI runners and build minutes

☺ Like you're 10: Handing every job the biggest, fastest machine wastes money per minute; handing every job the smallest, slowest one can waste just as much by making the job run long enough to cost more anyway.

Right-sizing a CI runner is the same idea as right-sizing a production VM, aimed at a different bill. A linter that finishes in 20 seconds gains nothing from an 8-core runner — it's paying a higher per-minute rate for cores it never touches. But the opposite mistake is just as real and less often caught: a CPU-bound compile or a large monorepo's full test run, forced onto an undersized runner, can take three or four times as long to finish — and because you're billed rate × minutes, a slow cheap runner sometimes costs more in total than a fast expensive one for the exact same job. There's a real optimum, and it's found by measuring, not guessing.

Runner strategyWhat it costsWhere it wins
Hosted, default sizePer-minute rate, zero ops overheadLow-to-moderate volume, small teams, no infra to run
Hosted, larger runnerHigher per-minute rate, roughly linear in vCPUCPU-bound jobs where the extra cores cut wall-clock time by more than the rate multiplier
Self-hosted, static fleetFixed infra cost, 24/7, whether busy or idleSteady, predictable, high-volume load that keeps the fleet genuinely busy most hours
Self-hosted, autoscaledInfra cost only while jobs are queued or runningBursty load — daytime PR traffic, quiet nights and weekends — where a static fleet would sit idle

The most common waste source in the self-hosted row isn't a wrong instance type — it's a fleet sized for peak PR volume that then runs at that size around the clock. A pool of twenty self-hosted runners provisioned for a Tuesday-afternoon crunch is mostly idle capacity every night and every weekend, and idle self-hosted compute bills exactly the same as busy self-hosted compute. Tools like actions-runner-controller (ARC) for GitHub Actions on Kubernetes, or GitLab's autoscaling Docker Machine and Kubernetes executors, scale the runner pool toward zero when no jobs are queued and back up when they are — turning a fixed 24/7 cost into one that roughly tracks actual pipeline activity. This is the same reconciliation idea covered in infrastructure as code — declare the desired pool size as a function of queue depth, and let a controller keep it converged — just pointed at build capacity instead of application capacity.

Caching: turning redundant builds into free ones

☺ Like you're 10: If nothing about a piece of your project changed, the pipeline shouldn't have to rebuild it from scratch — it should just reuse what it already made last time.

Right-sizing controls the price of a minute; caching controls how many minutes you need in the first place, and for most pipelines it's the bigger lever. There are three layers worth caching, and mature pipelines use all three:

# GitHub Actions dependency cache, keyed on the lockfile so a
# cache hit means "these exact dependencies were already installed."
- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
    restore-keys: |
      npm-${{ runner.os }}-

# Docker BuildKit registry cache — pull the last image's layers
# as a cache source instead of rebuilding every layer cold.
- uses: docker/build-push-action@v6
  with:
    cache-from: type=registry,ref=registry.acme.io/app:buildcache
    cache-to: type=registry,ref=registry.acme.io/app:buildcache,mode=max

Treat cache hit rate as a first-class pipeline metric, reported on the same dashboard as build pass rate — a monorepo's remote cache sitting at 40% hit rate is telling you exactly as much as a flaky test suite is; something in the cache key is too broad (invalidating on changes that don't actually affect the output) or too narrow (missing a real dependency and technically correct but useless). The Scaling CI/CD Across Teams page goes further into remote-cache design for large monorepos; here, the point is narrower — every redundant build a cache prevents is runner-minutes that never get billed at all, which beats right-sizing them cheaper.

⚠ Watch out

A cache is only as trustworthy as its key. A key that's too coarse — say, keyed on the branch name instead of a dependency hash — can silently serve stale build outputs across an unrelated change, and a test suite that passes against a stale cache isn't testing what actually shipped. Include the lockfile hash, the OS, and the toolchain version in every cache key, and treat an unexpectedly high cache hit rate on a PR that touched core dependencies as a bug to investigate, not a win to celebrate.

Per-team cost visibility for shared pipeline infrastructure

☺ Like you're 10: When twenty teams all push into the same shared build fleet, one invoice arrives for the whole fleet — and it takes real work to figure out whose pushes actually caused it.

This is the same shape of problem general cloud FinOps solves for a shared Kubernetes node — one bill, many tenants, no natural per-tenant line item — just one layer up the stack. A shared self-hosted runner pool, or a single org-wide GitHub Actions or GitLab CI/CD minutes quota, produces one aggregate number: this many runner-minutes, this many dollars, this month. It says nothing about which team's test suite, which repo's matrix, or which service's container builds actually consumed it, and "the pipeline is expensive" without a "because of whom" is not something anyone can act on.

Closing that gap starts with attribution, not a new tool. Tag every job with a team or cost-center label — a GitHub Actions runs-on label pool scoped per team, a GitLab CI job tags: field, or a Kubernetes namespace per team when self-hosted runners run on a cluster — and export usage against those labels. GitHub's Actions usage metrics and billing APIs, GitLab's CI/CD minutes usage reports (broken down per project and group), and self-hosted options like BuildBuddy's build analytics or a homemade export into Grafana all do the same underlying job: turn "the fleet cost $8,400 this month" into "payments spent $2,100, search spent $3,600, and $1,200 was genuinely shared CI infrastructure nobody should be blamed for." As with any billing feature, verify current export capabilities against your CI vendor's docs — this is an area where tooling has matured quickly and unevenly across GitHub, GitLab, and CircleCI.

Once teams can see their number, the same two allocation models general cloud FinOps uses apply here, and they trade off the same way. Showback — publishing each team's CI spend without moving any budget — is nearly always the right starting point: it creates awareness with no political cost, and it surfaces the inevitable attribution gaps (unlabeled shared workflows, a runner pool three teams quietly share) while nothing serious is riding on the number's accuracy. Chargeback — billing that spend against the team's actual budget — creates sharper accountability once the data is trusted, but it demands allocation accurate enough to survive an argument, because a chargeback number people don't believe just gets disputed instead of acted on.

The deployment-frequency vs. compute-spend trade-off

☺ Like you're 10: Shipping ten times more often is only "ten times more helpers, same total pizza" if you actually reuse the pizza that didn't change — otherwise it really is ten times the pizza.

This is where the DORA metrics and a delivery pipeline's cost curve intersect, and it's the connection most teams never make. Deployment frequency is one of DORA's four metrics, and elite performers deploy on-demand, multiple times a day. DORA's own finding — covered on that page — is that elite teams get both speed and stability from the same underlying practice: small, frequent, reversible changes, enabled by trunk-based development and heavy automated testing. What that page doesn't price in is the pipeline's own compute bill, and if your pipeline's design doesn't change alongside your deploy frequency, that bill scales close to linearly with it.

Here's the arithmetic. A team running one full pipeline a day — a 45-minute test suite on a 4-core runner, no caching, no selective testing — spends 45 runner-minutes daily. Move that same team to on-demand deploys, twenty a day, elite-tier by DORA's own benchmark, and a pipeline that still runs the full suite on every push spends 900 runner-minutes a day: a 20× throughput increase paid for with a 20× compute bill. Nothing about DORA's deployment-frequency metric objects to that outcome — the metric only measures how often you ship, never what each ship cost to build. That's a Goodhart's Law trap with a dollar sign on it, and it's the direct sibling of the gaming risk the DORA metrics page already warns about: a number chased without the underlying practice behind it stops being a good measure of anything.

The practices this page has already covered are exactly what breaks the 1-to-1 scaling. A small, trunk-based commit should only need to rebuild and retest the surface it actually touched — a cache hit on unaffected dependencies, a remote build cache skipping untouched packages, selective/impacted test targeting (the domain of Testing in the Pipeline) running only the tests a change could plausibly break. Model the same team with caching landing a realistic 70%+ hit rate and impacted-test selection trimming a typical small trunk commit's marginal run to about 8 minutes: the first run of the day still costs 45 cold minutes, but each of the next nineteen costs roughly 8, for a daily total near 197 runner-minutes — a 20× throughput increase for roughly a 4.4× compute increase, because the pipeline is finally exploiting the same "small, incremental change" property that makes frequent deploys safe in the first place.

Same 20× jump in deploy frequency — very different compute bill deploys per day CI runner-minutes / day 1 5 10 20 900 0 naive: 900 min/day cost-aware: 197 min/day both start cold: 45 min
◆ Key idea

Deployment frequency and CI compute spend are not opposites — the same discipline that decouples throughput from risk (small, trunk-based, well-tested changes, per the DORA research) is what decouples throughput from cost, if the pipeline is built to notice how small each change actually is. A team that raises deploy frequency without touching caching or test selection isn't proving deployment frequency is expensive — it's proving its pipeline never learned to tell a one-line change from a full rewrite.

Guardrails that keep the pipeline honest

☺ Like you're 10: A budget nobody checks isn't a budget — set a timer on every job, cancel the run that a newer push already made pointless, and save the expensive checks for the pushes that are actually about to ship.

The levers above compound; the guardrails below stop them from quietly eroding. Four are cheap enough to add to almost any pipeline this week:

# Cancel a superseded run the moment a newer commit lands on the
# same PR — and cap every job so a hang can't bill indefinitely.
concurrency:
  group: ci-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    timeout-minutes: 20
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test
⚠ Watch out

cancel-in-progress is exactly the kind of setting that should never reach a production-deploy job. Cancelling an in-progress test run to save a few runner-minutes is a clean win; cancelling an in-progress deployment mid-rollout because a newer commit landed can leave a release half-applied. Scope concurrency cancellation to pre-merge CI workflows specifically, and leave deploy workflows to queue behind each other instead — the guardrail that saves money on the left side of the pipeline is a genuine hazard on the right side of it.

All four guardrails share a spending-limit backstop worth setting regardless: GitHub Actions supports an org-level spending limit on top of included minutes, and GitLab lets you cap purchased CI/CD minutes per group — both exist specifically so a runaway matrix or an infinite retry loop hits a wall instead of an unbounded invoice. A budget that only shows up after the month closes is a postmortem; a budget enforced in the pipeline itself is a guardrail.

🎬 At the Ship-It Guild
🦫

Benny the Beaver: We hit it, Sol — twenty deploys yesterday. On-demand, no queue, no waiting for a release train. Elite tier!

🦥

Sol: Congratulations on the deploys. Now let me check the other number... the runner bill went from $40 a day to $760.

🦫

Benny the Beaver: That's... not the number I was hoping you'd say.

🦊

Foxy: Wait — DORA says fast deploys are supposed to be a good thing. Why's it costing so much?

🦥

Sol: Because the pipeline's still running the full matrix, cold, on every single push. DORA never promised that part was free — it only measures how often you ship.

🐢

Timmy the Turtle: So we cache what didn't change and only run the tests a commit could actually break. Same twenty deploys — a pipeline that's actually paying attention to what moved.

🦫

Benny the Beaver: ...I can have that live by Friday. Deploy frequency stays. The bill doesn't.

✓ Checkpoint

1. Why does this page scope itself to the delivery pipeline's own bill rather than general cloud FinOps — what's the actual difference in what's being measured? 2. Give the three-part formula behind a CI bill, and explain why a matrix build is the multiplier teams most often forget to check. 3. What's the risk of right-sizing a runner too small, not just too large? 4. Name the three layers of CI caching this page covers, and why "cache hit rate" deserves to be a tracked metric rather than a background detail. 5. Explain, in your own words, how a shared runner fleet recreates the same allocation problem as a shared Kubernetes node — and name the two ways teams typically report cost back once it's allocated. 6. Walk through the worked example: why does a naive pipeline's cost scale roughly 1-to-1 with deploy frequency, and what two practices flatten that curve?

Check your answers
  1. General cloud FinOps allocates the production bill — servers and infrastructure serving customers. This page covers the compute spent building, testing, and packaging code before it reaches production: CI runner minutes, which run constantly and are easy to overlook because no customer ever touches them directly.
  2. Cost = rate per minute × minutes run × number of parallel jobs. A matrix build is easy to forget because it turns one conceptual job into many billed jobs simultaneously — a 5-minute test suite across a 12-cell matrix is 60 runner-minutes, not 5, and that multiplication is invisible unless you go looking for it.
  3. An undersized runner can make a CPU-bound job run several times longer; because billing is rate × minutes, a slow cheap runner can end up costing more in total than a fast expensive one for the identical job — the goal is the actual cost optimum, not simply "always pick the cheapest instance."
  4. Dependency caching (package managers, keyed on the lockfile), Docker layer caching (registry/inline cache for container builds), and build-system remote caching (Bazel, Gradle, Turborepo, Nx Cloud — keyed per task on content hashes). Cache hit rate deserves tracking because a low hit rate means real, avoidable redundant builds are happening — the same way a low test pass rate signals a real problem, not background noise.
  5. A shared runner fleet (or a shared org-wide CI minutes quota) produces one aggregate bill with no built-in breakdown by team — exactly like one Kubernetes node's bill has no built-in breakdown by pod. Both need explicit attribution (labels/tags plus a usage export) to turn the aggregate into a per-team number. Teams report that number back via showback (visibility only, no budget impact) or chargeback (billed against the team's actual budget).
  6. A naive pipeline reruns its full, cold suite on every deploy, so total compute scales almost linearly with how many times you deploy — 20× more deploys means close to 20× more runner-minutes. Caching (skipping unchanged work) and selective/impacted test targeting (only running tests a small change could plausibly break) flatten that curve, because most of what a small, incremental deploy touches was already covered by the last run.