Hands-On Labs · The Capstone · Part 1 of 6

Capstone Part 1 — Define the SLOs

This is the first of six parts that build one continuous project: checkout-api, a small synchronous payments-adjacent service you'll monitor, page for, break, and recover over the rest of this capstone. Every later part assumes the numbers you produce here — Part 2 builds alerts off the exact SLOs you set today, Part 4's incident review is graded against the exact error budget you calculate today. Nothing here needs a cluster or a terminal; you need a service description, a calculator, and the discipline to write the arithmetic down before you argue about it. By the end of this page you'll have three concrete artifacts: a table of SLIs, a table of SLOs with the error budget each one produces, and a written error-budget policy that says exactly what happens the day that budget hits zero.

☺ Explain it like I'm 10

Imagine you're setting the rules for how many days you're allowed to be late delivering the school newspaper before the editor steps in. First you have to decide what "late" even means — is it "didn't print at all" or "printed but past the deadline"? That's picking your SLI. Then you pick the target — "printed on time at least 999 days out of 1000" — that's your SLO. Then you do the math on how many late days that actually allows you over a month, which is your error budget. And then, before you've ever actually been late, you write down what happens on the day you run out of late days — no negotiating in the moment, because you already agreed to it. That's the whole page: pick what to measure, pick the target, do the arithmetic, write the rule down in advance.

🦥Your host for this part: Sol the Sloth — the exact same arithmetic from SLIs, SLOs & error budgets, now run against a real service description instead of a worked example, slowly and correctly.
⚠ Where you're starting, and what you'll have when you're done

Starting: a plain-English description of checkout-api and nothing else — no monitoring, no alerts, no on-call rotation, no incident history. Leaving this page: a defined set of SLIs with exact numerator/denominator definitions, an SLO for each with its measurement window, the resulting error budget expressed both in minutes and in requests, and a written error-budget policy with a trigger, an enforcement action, and an exception path. Part 2 picks up exactly here and builds the dashboards and burn-rate alerts that watch the exact budget you calculate below.

What this part assumes, and what it produces

☺ Like you're 10: Just the reading you've already done and a calculator — nothing to install.

This part assumes you've read SLIs, SLOs & error budgets (the plain-language version of everything below) and, if you're studying toward the certification, Service Level Objectives & Error Budgets (the exam-focused version, including the exact worked checkout-API example this capstone continues). There's no tooling prerequisite — no cluster, no account, no CLI. What you need is a text editor and a willingness to do the percentage arithmetic by hand at least once before trusting a spreadsheet with it, the same habit Sol insists on. You'll produce exactly three artifacts, and every later capstone part cites them by name: an SLI table (what gets measured and how), an SLO table (the target and window for each SLI, with its error budget), and an error-budget policy (the written rule for what happens at zero). Keep all three somewhere you can paste from — Part 2 wires monitoring directly off the SLI table, and Part 3's runbook quotes the policy's trigger and exception path verbatim.

Meet checkout-api: the service you're defining SLOs for

☺ Like you're 10: A single web request that takes someone's cart, charges their card, and hands back a receipt — and every part of this capstone happens to that one request.

checkout-api is the last synchronous step in a purchase funnel. An authenticated client sends POST /v1/checkout with a cart ID; the service validates the cart against an inventory service, calls a third-party payment processor to authorize and capture the charge, writes an order row to its own database, and returns a response — 200 with an order confirmation on success, a 4xx for a genuinely invalid request (a declined card, an expired cart, a bad auth token), or a 5xx when checkout-api itself, or something it depends on, fails to do its job. It runs behind a load balancer across two availability zones and serves roughly 5,000,000 valid checkout requests over a rolling 30-day window — quiet at 2 a.m., spiking well past that average during flash sales. It sits directly on the revenue path: there's no retry queue behind it and no background job absorbing the failure. A failed checkout isn't a delayed checkout — it's an abandoned cart.

FactValueWhy it matters later
Servicecheckout-apiThe name every later capstone part refers back to
Entry pointPOST /v1/checkout, synchronous HTTPRules out freshness-style SLIs — the caller is waiting on the line
Traffic~5,000,000 valid requests / rolling 30-day windowTurns every percentage in this page into a request count
Dependenciesinventory service, external payment processor, order databasePart 4's incident and Part 5's capacity plan both trace back to one of these
SLI measurement boundaryThe load balancer, not inside checkout-api's own processMeasures what the client actually experienced, not internal component health
Owning teamCheckout ReliabilityThe name the error-budget policy below is written for
Client Load balancer availability + latency SLIs measured here checkout-api Inventory service Payment processor (external) Order database POST /v1/checkout

Choosing the SLIs: what a checkout-api user actually feels

☺ Like you're 10: Measure whether the receipt showed up and how fast — not the CPU inside the machine that printed it.

Applying the "few, request-based, user-visible" criteria from SLIs, SLOs & error budgets to checkout-api's description above rules out most of what's easy to instrument. Database connection-pool saturation and payment-processor round-trip time are real operational signals — worth alerting on internally — but neither one is something the client on the other end of the HTTP connection experiences directly. Two SLIs survive the filter, both measured at the load balancer:

SLIGoodValid (denominator)
AvailabilityResponse is not a 5xxAll checkout requests reaching the load balancer, excluding synthetic health checks and requests that failed auth before reaching checkout-api at all
LatencyResponse completes in under 300ms end-to-endSame denominator as availability, restricted to requests that got a response at all (a timeout counts against latency, not as an exclusion)

Notice what's deliberately excluded from the numerator's failures: a 4xx for a declined card or an expired cart isn't counted against availability, because checkout-api did its job correctly — it told a real customer the truth about a real business condition. Counting legitimate declines as "bad" would push the team to chase a number that has nothing to do with whether the service is actually working, exactly the kind of denominator mistake the foundations page warns about.

⚠ A third candidate SLI that doesn't belong in this table

"Percentage of checkout requests that don't result in a duplicate or missing payment charge" is a real, measurable signal — and it does not get an SLO in the table below. Payment-correctness bugs aren't something a team should ever be "allowed" to spend budget on the way a few minutes of downtime can be spent. Treat it instead as a zero-tolerance invariant: any confirmed double-charge is an immediate incident regardless of how much error budget remains elsewhere, gets refunded automatically, and gets the full blameless postmortem treatment no error-budget policy would ever excuse. If this capstone's simulated incident in Part 4 ever involves a payment defect, this is the invariant it will have broken — not the availability SLO defined below.

Setting the SLOs: the target and the window

☺ Like you're 10: Pick a goal loose enough that nobody notices you hitting it, tight enough that everybody would notice if you didn't.

Both SLOs use the same rolling 30-day window as the worked example in Service Level Objectives & Error Budgets, so the arithmetic below lines up exactly with material you may already have memorized for the exam. The two targets are deliberately unequal:

SLISLOWindowReasoning
Availability99.9%Rolling 30 daysAn abandoned cart from a hard failure is the worst outcome on this service; three nines is the tightest target the team can hold without over-investing in marginal reliability past what a shopper would ever notice
Latency99.5% under 300msRolling 30 daysDeliberately looser than availability — an occasional slow-but-successful checkout is a worse experience than none at all only in the extreme tail, so this SLO tolerates five times the "badness rate" of availability before it's in trouble
◆ Key idea

Nothing above 99.9% was seriously considered for availability, and that's the point, not an oversight. The foundations page's rule — the right SLO is the loosest one users won't notice you missing — applies literally here: 99.99% would shrink the error budget by 10x for a service whose users have never once complained about a monthly outage measured in tens of minutes. That saved budget is what funds the deliberate risk-taking — faster ships, a chaos experiment in Part 6 — that a tighter SLO would have quietly outlawed.

Doing the arithmetic: the error budget in minutes and requests

☺ Like you're 10: Subtract your goal from 100%, multiply by how many minutes are in the month, and that's exactly how many minutes you're allowed to be broken.

A rolling 30-day window is 43,200 minutes (30 × 24 × 60). Applying Error budget = (100% − SLO) × window to the availability SLO above:

Availability SLO:      99.9%
Allowed failure rate:  100% − 99.9% = 0.1%
30-day window:          30 × 24 × 60 = 43,200 minutes

Error budget (time):   0.1% × 43,200 min  =  43.2 minutes / month
Error budget (requests): 0.1% × 5,000,000 =  5,000 failed requests / month

Both framings describe the same 0.1%, but they answer different questions and only agree under an assumption worth stating out loud: that failures are spread roughly evenly across the traffic curve. 43.2 minutes answers "how long could checkout-api be completely down before the budget is gone." 5,000 failed requests answers "how many individual checkouts could fail while the service stays technically up," which is the more realistic failure mode — a flaky downstream dependency that fails 2% of requests for six hours burns requests, not minutes of total outage, and a policy that only watches the minutes framing would miss it entirely. Part 2's alerting is built on the request-based framing for exactly this reason; the minutes framing is what you'll quote to non-engineering stakeholders because it's the one people can picture.

The latency SLO doesn't convert to a clean minutes figure at all — "minutes of slowness" isn't a coherent quantity the way "minutes of down" is — so its budget is native to the request framing:

Latency SLO:           99.5% under 300ms
Allowed miss rate:     100% − 99.5% = 0.5%
Error budget (requests): 0.5% × 5,000,000 = 25,000 slow requests / month
⚠ Watch out

Don't average the two budgets together, and don't let a service that's spent its latency budget count as "fine" because availability still has room. These are two separate SLOs with two separate budgets and, in Part 2, two separate burn-rate alerts. A checkout-api that's 100% available but consistently slow has a real, unaddressed reliability problem even while its availability number looks perfect.

Writing the error-budget policy

☺ Like you're 10: Write down, before anyone's stressed, exactly what happens on the day the late-days run out — so nobody has to argue about it in the moment.

Per Service Level Objectives & Error Budgets, a real policy specifies three things: the trigger, the enforcement action, and an exception path. Here's checkout-api's, scoped to the availability SLO (the one with the cleanest minutes framing to alert against):

ERROR-BUDGET POLICY — checkout-api
Owner: Checkout Reliability team · Reviewed quarterly
SLO in scope: Availability, 99.9%, rolling 30-day window (43.2 min / 5,000 req budget)

TRIGGER 1 — early warning
  ≤25% of budget remains (≤10.8 min / ≤1,250 failed requests left)
  with more than 5 days remaining in the window.
  -> Post a heads-up in #checkout-reliability. No freeze yet.

TRIGGER 2 — freeze
  Budget fully consumed (0 minutes / 0 requests remaining)
  before the 30-day window closes.
  -> FREEZE: no new checkout-api feature launches, no non-critical
     dependency upgrades, no checkout-flow experiments.
  -> REDIRECT: the owning team's next sprint goes to reliability
     work tied to whatever actually burned the budget — not
     general cleanup chosen after the fact.
  -> RESUME: automatically, once the rolling 30-day window has
     aged the bad minutes out and the trailing budget is > 0 again.

EXCEPTION PATH
  A change may still ship during a freeze only with written
  sign-off from the on-call incident commander or the team's EM,
  logged in the incident channel, and limited to: security
  patches, active-incident mitigations, legally mandated fixes.

OUT OF SCOPE
  Payment-correctness incidents (duplicate/missing charges) are
  never "budget" — see the zero-tolerance note above. They trigger
  an incident regardless of this policy's state.

Two details in that document are doing more work than they look like. The early-warning trigger fires at 25% remaining, not at zero — a policy that only reacts once the budget hits exactly zero gives the team no runway to redirect before the freeze is mandatory. And the exception path names a specific role (the on-call IC or the EM), not "use your judgment" — a policy without a named approver quietly becomes optional the first time someone argues their launch is important enough to be the exception, which is precisely the failure mode the foundations page and the domain page both warn about under "just lower the SLO instead."

What "done" looks like for Part 1, and where Part 2 picks up

☺ Like you're 10: Three documents in hand — what to measure, the goal, and the rule for when the goal is missed — and every later part of this build reads straight from them.

At the end of this part you have exactly the three artifacts promised at the top: the SLI table (availability and latency, both measured at the load balancer, both with an explicit numerator and denominator), the SLO table (99.9% and 99.5% respectively, both over a rolling 30-day window, with budgets of 43.2 minutes / 5,000 requests and 25,000 requests), and the written error-budget policy above. Nothing here gets thrown away — each later part reads one of these artifacts directly:

PartWhat it does with today's artifacts
2 — Build the Monitoring & AlertingTurns the SLI definitions into real queries and the request-based budgets into multi-window burn-rate alerts
3 — Write the On-Call RunbookQuotes this page's policy trigger and exception path verbatim as the runbook's escalation section
4 — Run the Incident & Write the PostmortemSpends real minutes against the 43.2-minute budget calculated above, on purpose, as part of the scenario
5 — Capacity Plan & Load TestLoad-tests against the 5,000,000-request traffic figure from today's world-model table
6 — Chaos Engineer ItDeliberately spends a bounded slice of whatever budget remains, exactly as the policy above permits
🎬 At the Reliability Watch
🦥

Sol the Sloth: ...Ninety-nine point nine percent, forty-three thousand two hundred minutes in the window... comes out to forty-three point two minutes. Checked it twice.

🦊

Foxy: Forty-three minutes feels thin for a checkout flow. Why not just target 99.99% and sleep better?

🦥

Sol the Sloth: That would leave us about four minutes instead of forty-three. Nobody's ever complained about a monthly outage measured in tens of minutes — tightening the target past what users notice just deletes the budget we'd use for everything else.

🐢

Timmy the Turtle: Then I care more about the policy than the target. When that budget actually hits zero, who's allowed to say "ship anyway," and does it happen anywhere but the incident channel?

🦥

Sol the Sloth: On-call IC or the EM, in writing, logged. Not a hallway conversation.

🦉

Professor Owl: Then it's a real policy, not a suggestion. Write it down now, while nobody's under pressure — that's the entire discipline.

✓ Checkpoint

1. Why does checkout-api's availability SLI exclude 4xx responses caused by a declined card, and why is that different from excluding a 5xx? 2. With a 99.9% availability SLO over a rolling 30-day window (43,200 minutes), what's the exact error budget in minutes, and in failed requests against 5,000,000 monthly traffic? 3. Why does the "duplicate or missing payment charge" signal get treated as a zero-tolerance invariant instead of a third SLO in this page's table? 4. Name the three things a real error-budget policy must specify, and explain what an exception path without a named approver quietly turns into.

Check your answers
  1. A declined card is checkout-api correctly reporting a true business condition — the service did its job. A 5xx means the service (or a dependency it relies on) failed to do its job. Counting legitimate declines as "bad" would make the SLI chase customer behavior instead of measuring the service's own correctness.
  2. (100% − 99.9%) × 43,200 = 0.1% × 43,200 = 43.2 minutes. In requests: 0.1% × 5,000,000 = 5,000 failed requests over the same 30-day window.
  3. Because a double-charge or missing charge has direct monetary and trust cost per occurrence, and a team should never be implicitly "permitted" to let a few happen just because an unrelated availability budget still has room. It's handled as an always-incident invariant, independent of any SLO's remaining budget.
  4. A trigger (the condition that fires the policy, ideally including an earlier warning threshold, not just zero), an enforcement action (freeze risky changes, redirect engineering time to reliability work), and an exception path (who may still ship during a freeze, and under what narrow conditions). Without a named approver, the exception path becomes whoever argues hardest in the moment gets to override it — which is functionally no policy at all.

Part 1 gave you a defined SLI, a defined SLO, an exact error budget, and a written policy for what happens when it's spent — the four things every remaining part of this capstone keeps referring back to by name. Continue to Capstone Part 2 — Build the Monitoring & Alerting, where these exact SLIs become real queries and this exact budget becomes a multi-window burn-rate alert (see multi-window, multi-burn-rate alerting for the theory behind it). Or step back to Run a Reliable Service — start here for how this capstone's six parts fit together, drill the arithmetic itself under timed conditions in Drill — SLO & Error-Budget Calculation, or read how Google's own team handled the first real launch their error-budget policy froze in Google & the error-budget policy.