Multi-Window, Multi-Burn-Rate Alerting

SLIs, SLOs & error budgets gave you a number worth protecting — a spendable allowance of allowed failure over a rolling window. This page answers the question that number leaves open: how do you actually get paged before it's gone, without getting paged for nothing every time the number wobbles? Google's SRE Workbook answers it with a specific, load-bearing pattern — multi-window, multi-burn-rate alerting — that every serious SLO-based paging system, hand-rolled or vendored, is some variation of. We'll build it from first principles: why one threshold can't do the job, what a burn rate actually is, why every alert needs two windows instead of one, how the four standard severity tiers are derived, and a full worked PromQL example extending the checkout API from the SLO page.

Why a single error-rate threshold fails

☺ Like you're 10: One alarm that just says "too many errors" can't tell a five-minute hiccup from a two-day slow leak — so it either screams at nothing or stays silent until it's already too late.

You have an SLO and the error budget it implies — say 99.9% over a rolling 30-day window, which SLIs, SLOs & error budgets works out to 43.2 minutes of allowed downtime. The obvious next move is to alert when the budget looks threatened. Two naive approaches both fail, in opposite directions, and understanding why each fails is what motivates everything else on this page.

Alert on the SLO's own compliance window. If you evaluate "is the 30-day error rate above 0.1%?" and page when it trips, the alert is maximally precise — it only fires when the SLO is genuinely, provably breached — but it's also useless as an early warning. By the time a 30-day rolling average crosses the line, the budget is already fully spent; there was no moment before the page where a human could have intervened and changed the outcome. You've built a very accurate obituary, not an alarm.

Alert on a short window's raw rate. If you instead evaluate "is the error rate over the last 5 minutes above 0.1%?", you get near-instant detection — but 0.1% is a long-run target, not a value real traffic holds steady at minute to minute. Any service's 5-minute error ratio bounces above and below its long-run average constantly from ordinary variance, deploys, and small transient blips that self-resolve. Set the threshold at the SLO's raw value and you page constantly for noise; loosen it enough to stop the noise and you've quietly given up the ability to catch anything but the most catastrophic failures.

The SRE Workbook frames this as two properties in tension: detection time (how fast do you find out?) and precision (what fraction of alerts correspond to something real?). Short windows buy detection time and spend precision; long windows buy precision and spend detection time. A single window, at a single threshold, can only ever sit at one point on that trade-off curve. Multi-window, multi-burn-rate alerting is the answer that gets both properties at once, by refusing to make one window do both jobs.

Burn rate: normalizing error rate against your budget

☺ Like you're 10: Burn rate is just "how many times faster than the sustainable pace am I using up my allowed failures right now" — 1x is exactly on pace, 14.4x means you'd empty the whole month's budget in about two days if it kept up.

A raw error percentage doesn't tell you anything about urgency on its own — 1% errors is catastrophic for a service with a 99.99% SLO and comfortably survivable for one with a 99% SLO. Burn rate fixes this by normalizing the observed badness against the budget itself:

burn rate (BR) = observed bad-event ratio / (1 − SLO)

BR = 1   →  consuming budget at exactly the sustainable rate —
            sustained for the full compliance window, you land at
            precisely 0% budget remaining right as the window ends
BR = N   →  consuming budget N times faster than sustainable

fraction of budget consumed over time T at burn rate BR:
    consumed(T) = BR × T / compliance_window

time to fully exhaust a full budget at a sustained burn rate BR:
    time_to_exhaustion = compliance_window / BR

Worked example, continuing the checkout API's 99.9% SLO over a 30-day window from SLIs, SLOs & error budgets: 1 − SLO = 0.1%. If the checkout API's current error ratio is 1.44%, its burn rate is 1.44% / 0.1% = 14.4. At that rate, the full 30-day budget would be gone in 30 / 14.4 ≈ 2.08 days — about 50 hours — if nothing changed. That single number, 14.4, is doing real work: it's the same threshold regardless of whether the underlying SLO is 99.9% or 99.99%, because it's already normalized against whatever budget that SLO implies. This is what lets a burn-rate alert rule travel unmodified across services with completely different SLO targets, as long as each service's own 1 − SLO is plugged into its own alert.

Pairing a long window with a short window: solving slow reset

☺ Like you're 10: A single long-window alarm is accurate, but once the fire's out it keeps ringing for ages, because it's still averaging in the bad minutes from before — so you glue a fast, twitchy alarm next to it that shuts up the instant things are actually fine again.

Burn rate solves the normalization problem, but not the detection-time-versus-precision trade-off — that still lives entirely in which window you evaluate the burn rate over. A burn-rate alert on a long window (say, the last hour) has good precision: an isolated two-minute blip barely moves a full hour's average, so it won't falsely trip. But it has a second, less obvious problem — slow reset. Suppose a real 12-minute outage happens and gets fixed. The trailing 1-hour window still contains those 12 bad minutes for the rest of that hour, so the alert keeps firing — paging on an incident that's already resolved — for up to 48 more minutes after the actual fix landed. That's not a false positive exactly, but it's a stale one, and it trains whoever's on call to distrust the page the moment it fires.

Google's fix is to never evaluate a burn-rate alert against one window alone. Every alert pairs a long window (the primary detection window, sized for precision) with a short window — conventionally about ¹⁄₁₂ the length of the long one — and requires both to be breaching the same burn-rate threshold at once, joined with a logical AND, not OR. During a genuine, ongoing incident, both windows run hot together. The moment the incident is actually fixed, the short window — which only looks back a few minutes — clears almost immediately, and because the AND condition needs both windows lit, the alert clears with it, even though the long window's trailing average is still technically contaminated by the resolved incident for a while longer. The short window is what makes the alert self-healing on its own schedule instead of on the long window's schedule.

The four standard tiers: multi-burn-rate

☺ Like you're 10: One twitchy pair of alarms for "this is bad right now, wake someone up" and a second, more patient pair for "this has been leaking a little too much for a little too long — file a ticket."

A single (long window, short window, multiplier) triple still only covers one point on the urgency spectrum. A budget draining fast enough to empty in two days deserves a page at 2 a.m.; a budget draining just slightly faster than sustainable, in a way that would only empty it after the full 30 days, deserves a ticket someone picks up Monday, not a 2 a.m. page. Google's Workbook lays out four tiers that cover both ends, each pairing a long window with a short window at a fixed ¹⁄₁₂ ratio, all four tied to fixed error-budget-consumption percentages rather than arbitrary-looking multipliers:

Read the table as two matched pairs. The 1h / 6h long windows are the page tier — anything burning fast enough to matter within days needs a human paged now, and both rows use the short-window trick from the previous section for fast reset. The 24h / 3d long windows are the ticket tier — burns slow enough that the full 30-day budget would still last well over a week, which doesn't justify a 2 a.m. page but absolutely justifies a queued item, since left unaddressed it will eventually breach the SLO. Notice the last row's multiplier is exactly 1× — that's not a typo; it's the alert for "we are burning budget at precisely the boring, sustainable, break-even rate, and have been for three straight days," which is worth a ticket precisely because three days of even nominal burn is a real, non-noise signal, not a blip.

Treat this exact table as the community's widely-used starting template, adapted from the Google SRE Workbook's alerting chapter — not a universal constant. Verify the current numbers against Google's own documentation if you need to cite them precisely, and expect to retune the percentages and windows for your own service's traffic pattern and your team's on-call tolerance, which is exactly the next section.

Sizing burn-rate multipliers against a target pages-per-quarter

☺ Like you're 10: Don't just guess at the thresholds — replay them against the last few months of real data and count how many times each one would have actually gone off, then tighten or loosen until that count matches how often you're willing to be woken up.

The four-row table is a starting point, not a delivered answer, because the right multiplier depends on a number no formula can supply: how many legitimate pages per quarter your team is willing to absorb from this one service. Google's own guidance is to work the sizing problem backward from that tolerance rather than forward from the formula alone:

• Backtest against historical SLI data. Take each candidate (long window, short window, multiplier) triple and replay it against several months of the service's real error-ratio history. Count how many times it would have fired. If a candidate would have paged twenty times last quarter for events nobody remembers being real incidents, the multiplier is too tight for that service's normal noise floor.
• Validate it still catches what matters. The failure mode of tuning purely for silence is deafness — a threshold loosened until it never pages is worthless even if it never wakes anyone up. Cross-check candidate thresholds against your own past real incidents (see the worked incident case study): every incident that materially threatened the SLO should have tripped at least the page tier, and reasonably promptly.
• Work the formula in reverse for a target detection time. If you need to catch a complete, 100%-failure outage within 5 minutes, your fastest tier's short window must be ≤ 5 minutes — which is exactly why Google's fast tier uses a 5-minute short window paired with a 1-hour long window; the short window sets your floor on detection speed regardless of what the long window is doing.
• Re-derive, don't reuse, when the SLO changes. The 2% / 5% / 10% / 10% consumption percentages are what's actually fixed by policy; the 14.4 / 6 / 3 / 1 multipliers are downstream of them and of the 30-day compliance window specifically. Move to a 7-day rolling SLO window (see SLO windows & composite SLOs) and every multiplier in the table needs recomputing from the same formula — the percentages carry over, the multipliers do not.

This backtesting discipline is also the antidote to alert fatigue creeping back in through the multi-burn-rate door — see alert design & alert fatigue for the broader design philosophy this page's math serves, and monitoring & observability for why every one of these four alerts should still be symptom-based, not cause-based, underneath the burn-rate framing.

A worked PromQL example: the checkout API's 99.9% SLO

☺ Like you're 10: Here's the actual code — Ellie's recipe for turning "99.9% of checkout requests should succeed" into four real alerts that a pager can act on.

Extending the checkout API's 99.9% availability SLO from SLIs, SLOs & error budgets and the request-based SLI convention from monitoring & observability, here's the full four-tier rule set: recording rules that compute the error ratio at each of the eight required windows, then four alerts pairing them per the table above. 1 − SLO = 0.001 throughout.

groups:
  - name: checkout-slo-burn-rate
    rules:
      # --- error ratio, recorded at each window the alerts below need ---
      - record: sre:checkout_requests:error_ratio5m
        expr: |
          sum(rate(http_requests_total{route="/checkout",code=~"5.."}[5m]))
          /
          sum(rate(http_requests_total{route="/checkout"}[5m]))
      - record: sre:checkout_requests:error_ratio30m
        expr: |
          sum(rate(http_requests_total{route="/checkout",code=~"5.."}[30m]))
          /
          sum(rate(http_requests_total{route="/checkout"}[30m]))
      - record: sre:checkout_requests:error_ratio1h
        expr: |
          sum(rate(http_requests_total{route="/checkout",code=~"5.."}[1h]))
          /
          sum(rate(http_requests_total{route="/checkout"}[1h]))
      - record: sre:checkout_requests:error_ratio2h
        expr: |
          sum(rate(http_requests_total{route="/checkout",code=~"5.."}[2h]))
          /
          sum(rate(http_requests_total{route="/checkout"}[2h]))
      - record: sre:checkout_requests:error_ratio6h
        expr: |
          sum(rate(http_requests_total{route="/checkout",code=~"5.."}[6h]))
          /
          sum(rate(http_requests_total{route="/checkout"}[6h]))
      - record: sre:checkout_requests:error_ratio24h
        expr: |
          sum(rate(http_requests_total{route="/checkout",code=~"5.."}[24h]))
          /
          sum(rate(http_requests_total{route="/checkout"}[24h]))
      - record: sre:checkout_requests:error_ratio3d
        expr: |
          sum(rate(http_requests_total{route="/checkout",code=~"5.."}[3d]))
          /
          sum(rate(http_requests_total{route="/checkout"}[3d]))

      # --- alerts: same multiplier on both windows, joined with AND ---
      - alert: CheckoutErrorBudgetBurnFast
        expr: |
          sre:checkout_requests:error_ratio1h  > (14.4 * 0.001)
          and
          sre:checkout_requests:error_ratio5m  > (14.4 * 0.001)
        for: 2m
        labels: { severity: page }
        annotations:
          summary: "checkout burning error budget at 14.4x — 2% of the 30-day budget in 1h"

      - alert: CheckoutErrorBudgetBurnModerate
        expr: |
          sre:checkout_requests:error_ratio6h  > (6 * 0.001)
          and
          sre:checkout_requests:error_ratio30m > (6 * 0.001)
        for: 2m
        labels: { severity: page }
        annotations:
          summary: "checkout burning error budget at 6x — 5% of the 30-day budget in 6h"

      - alert: CheckoutErrorBudgetBurnSlow
        expr: |
          sre:checkout_requests:error_ratio24h > (3 * 0.001)
          and
          sre:checkout_requests:error_ratio2h  > (3 * 0.001)
        for: 15m
        labels: { severity: ticket }
        annotations:
          summary: "checkout burning error budget at 3x — 10% of the 30-day budget in 1d"

      - alert: CheckoutErrorBudgetBurnSlowest
        expr: |
          sre:checkout_requests:error_ratio3d  > (1 * 0.001)
          and
          sre:checkout_requests:error_ratio6h  > (1 * 0.001)
        for: 1h
        labels: { severity: ticket }
        annotations:
          summary: "checkout burning error budget at 1x (nominal) — 10% of the 30-day budget in 3d"

Two implementation notes worth the extra minute. First, notice the ticket-tier for: values (15m, 1h) are deliberately longer than the page tier's (2m each) — that's fine specifically because ticket-tier alerts aren't blocking anyone's sleep, so a little extra debounce against noise is pure upside there in a way it isn't on the page tier. Second, avg_over_time() of an already-computed ratio is not the same number as recomputing sum(errors)/sum(total) over the wider window unless traffic volume is roughly constant across that window — averaging pre-computed ratios silently misweights low-traffic periods. The recording rules above sidestep that trap by recomputing the true ratio at each window directly rather than averaging a narrower one, at the cost of Prometheus scanning more raw samples for the 24h and 3d rules. For a handful of services this is a non-issue; past a few dozen SLOs, most teams reach for a generator — Sloth produces exactly this rule shape from a single declarative SLO spec, and is worth adopting before you're hand-maintaining dozens of copies of the YAML above.

Where this fits

☺ Like you're 10: The math on this page decides when to page; what happens the moment after that page fires is a different lesson entirely.

Multi-window, multi-burn-rate alerting is deliberately narrow in scope — it answers exactly one question, precisely: given an error budget, when does spending it warrant interrupting a human, and how urgently? It says nothing about what that human should do once paged (incident management & on-call), what gets written up afterward (postmortems & blameless culture), or how you'd deliberately trigger the fast-burn tier on purpose to prove the whole chain works before a real incident does it for you (chaos engineering). It's also the concrete mechanism behind the abstract error-budget policy Google popularized — see Google & the error-budget policy for how the org-level policy and this page's per-alert math connect. If you want to build and test a rule set like the one above end to end, Capstone Part 2 — build the monitoring & alerting is exactly that exercise, and the alert-design drill is a faster, standalone rep of the same skill.