Operating DevOps · Best Practices & the Operating Model

Best Practices & the Operating Model

Every trap on this course's companion page — see DevOps anti-patterns & pitfalls — has a working alternative that real teams actually run on an ordinary Tuesday, not just in a values deck. This page is that alternative. Three mechanisms carry most of the weight: an org shape that routes a change through self-service instead of a queue, a postmortem process that stays blameless under real pressure instead of just on the cover page, and an error budget that turns "can we ship today" into arithmetic both Dev and Ops already agreed to respect. Nothing below is aspirational — each section is written the way a team actually operates it, including the parts that are easy to get half-right.

☺ Explain it like I'm 10

Picture a school where every hallway pass needs a trip to the principal's office — kids queue outside the door all morning, even just for the bathroom, because one office approves everything for the whole school. A better-run school keeps a stack of pre-approved passes right on each teacher's desk, so a kid goes the second they need to, no queue, no principal involved. When a window breaks anyway, that same school doesn't spend the meeting hunting for which kid to blame — it spends five minutes figuring out how the ball got over the fence in the first place, and fixes the fence. And every class gets an honest number of "free" minutes it's allowed to be a little wild before recess gets stricter — plenty left, try the cartwheel; almost gone, walk quietly to the swings. No principal's office, no witch hunt, just a fence that gets fixed and a number everyone already agreed to.

🦉🦊Your hosts for this topic: Professor Owl & Foxy — Owl draws the org shape that keeps ownership honest instead of routing it through a queue, and Foxy makes sure the postmortem that follows is genuinely blameless and not blame with extra paperwork.

Team topology: the org chart is part of the architecture

☺ Like you're 10: Some teams build the thing customers actually use, and one thin team builds the road underneath all of them — the road team's whole job is making everyone else's life easier, never approving their every move.

Culture & collaboration already established Conway's Law: your system ends up shaped like your org chart, whether or not anyone planned it that way. Team Topologies (Matthew Skelton and Manuel Pais) turns that observation into a design vocabulary. It names four team shapes — stream-aligned (owns one product or service slice end to end), platform (provides internal, self-service capability the stream-aligned teams build on), enabling (coaches a team through a new capability, then leaves), and complicated-subsystem (deep specialists for one genuinely hard piece) — and three ways teams interact: collaboration, facilitating, and X-as-a-Service. The healthy default at any size is a stream-aligned team practicing "you build it, you run it" — it owns its own pipeline, its own infrastructure, and its own pager. A platform or enabling team is not mandatory scaffolding you stand up on day one; it earns its place only once two or more stream-aligned teams are independently solving the same infrastructure problem, at which point a thin platform team paying that cost once, on everyone's behalf, is cheaper than paying it three separate times.

Team typeWhat it ownsWhen it should exist
Stream-alignedOne product or service slice, end to end — code, deploy, on-callAlways. This is the default shape; every other team type exists only to make it faster.
PlatformA self-service capability stream-aligned teams pull from — CI templates, a paved deploy path, a shared clusterOnce several teams are independently rebuilding the same plumbing
EnablingNothing permanent — coaching a team through a new skill or tool, then stepping back outA time-boxed engagement, not a standing team requests route through forever
Complicated-subsystemOne genuinely hard piece — a matching engine, a codec — needing deep specialistsRare. Most organizations never need one.

The interaction mode matters as much as the team type. Collaboration is expensive, high-bandwidth, and temporary by design — two teams working shoulder to shoulder while genuine discovery is still happening. Facilitating is what an enabling team does — active coaching with an explicit end date. X-as-a-Service is the mode a platform team should be aiming for on almost everything it offers: a stream-aligned team consumes a self-service API or template the same way it would consume a cloud provider's service, with no meeting, no ticket, and no human standing between the request and the result.

The anti-pattern this page exists to counter: the gatekeeper team

☺ Like you're 10: A gatekeeper team sounds responsible — someone's watching every change! — but it's really just one office everyone has to line up outside of, no matter how small the request.

What is DevOps? warned about this exact failure early in the course: a well-meaning reorg stands up a "Platform" or "Release Engineering" team, and within a year every other team routes deploys and infrastructure requests through it — recreating the ticket-and-wait handoff DevOps exists to remove, just with a friendlier name on the door. A gatekeeper team — often called Release Engineering, Change Management, or a Change Advisory Board — is the concrete shape that failure takes: a single, centralized team that must review and approve every production change, regardless of the change's actual risk. It is seductive precisely because it looks like governance. It fails for a structural reason, not a people reason: the gate has none of the context the team that wrote the change has, so it can't meaningfully assess risk — it can only make everyone wait the same amount of time whether the change is a one-line config flip or a schema migration. One queue serving every team in the company does not scale with headcount; it gets slower exactly when the organization is growing fastest and needs speed the most. And because Ops now owns the approval and Dev owns the code, it reinstates the exact incentive split — Dev pushing for speed, the gate holding for safety — that the wall of confusion was named to describe in the first place. DevOps anti-patterns & pitfalls catalogs this trap and several others like it in full; this page is the working alternative.

The gatekeeper team (anti-pattern) one queue approves every change Team A Team B Team C Change Advisory Board reviews every release days of wait Production Stream-aligned + thin platform team self-service, same team's on-call Team A Team B Team C Platform team — paved path self-service APIs · CI templates no queue Production

A platform team just thin enough to stay a paved road

☺ Like you're 10: The road crew's whole job is carrying the boring, heavy stuff — permits, drainage, paving — so the delivery drivers can just drive, not build the road themselves every trip.

The reason a platform team exists at all is cognitive load: some of a stream-aligned team's mental effort is intrinsic (the actual business problem they're paid to solve), some is germane (learning that genuinely compounds), and a large share is extraneous — the accidental toil of wiring up a database, a pipeline, TLS, and a dashboard before a single line of feature code ships. A good platform team's entire job is absorbing that extraneous share, and it does that by publishing golden paths (see the glossary) that stream-aligned teams pull from, not by inserting itself into anyone's release. Two rules keep a platform team thin instead of quietly turning back into a gatekeeper: it earns each capability from actual repeated demand rather than building ahead of it, and it always ships an escape hatch — a team with a genuine edge case can step off the paved path and own more of the wiring itself, rather than being trapped by it. Consuming the platform should look like consuming a cloud service, not filing a request:

# .gitlab-ci.yml — a stream-aligned team's pipeline, consuming the
# platform team's golden-path template with zero tickets filed
include:
  - project: 'platform/golden-paths'
    ref: v4
    file: '/templates/service-deploy.yml'

variables:
  SERVICE_NAME: checkout-api
  SLO_TARGET: "99.9"

# Build, scan, canary deploy, dashboards, and on-call routing are all
# wired up by the include above. Any stage can still be overridden
# locally in this file — that's the escape hatch, not a special request.
◆ Key idea

The health check for a platform team is the same health check for the gatekeeper team it must never become: could a new engineer provision a pipeline and a deploy path without opening a ticket or waiting on a human — and could the platform team be on holiday the entire time? If yes, it's a paved road. If every request still routes through a queue, it's ticket-ops wearing a platform team's job title. This is exactly the boundary from DevOps to platform engineering walks in more depth, and scaling CI/CD across teams covers the pipeline-template mechanics shown above.

Blameless postmortems: the mechanics that keep "blameless" honest

☺ Like you're 10: Blameless doesn't mean nobody writes anything down — it means the write-up asks "what let this happen" instead of "whose fault was it," and someone actually checks that the fix gets done.

Culture & collaboration already defined blameless culture and why a generative organization produces it. This section is the part that turns that definition into a repeatable meeting: the practice traces back to Etsy's engineering team around 2012, where John Allspaw — the same engineer behind the 2009 Flickr talk this course opened with — argued that punishing the person closest to a failure only teaches everyone else to hide information, drawing on researcher Sidney Dekker's work on "just culture." A postmortem earns the word "blameless" through specific mechanics, not a mission statement at the top of the document:

The case study shows this pattern applied to one real incident end to end, and Etsy & continuous deployment covers the culture that produced it in more depth.

⚠ Watch out — postmortem theater

The most common way "blameless" fails in practice isn't a shouting match — it's quieter than that. The document has all the right section headers, the meeting follows the script, and nobody raises their voice. But the action items never get prioritized against the next sprint's feature work, so the same failure mode recurs in four months. Or the meeting itself is calm, and then a manager pulls the engineer aside afterward, one-on-one, to ask what they were thinking. Either version teaches the team the same lesson — that the safety was theater — and the next postmortem gets a little less honest than this one was. The only reliable test: pull up last quarter's postmortems and check how many action items actually closed.

Error budgets as an operating ritual, not a wiki page

☺ Like you're 10: The "how risky can we be today" number only works if someone actually looks at it every week and everyone agreed, in writing, what happens when it runs low.

This page assumes you already have the mechanics — SLIs, SLOs, the error-budget formula, and burn rate — from SLOs, Error Budgets & Toil; go there first if "43.2 minutes of allowed downtime" doesn't already mean something to you. What that page doesn't fully cover is how the number becomes a standing part of how an organization actually runs, week over week, rather than a Prometheus rule nobody looks at until an incident forces the question. Three habits make the difference. First, the policy is a written, jointly-owned document — signed off by engineering and whoever owns the roadmap, agreed on while the budget is healthy, so nobody is negotiating the rule for the first time during the incident that's about to exhaust it. Second, budget status is a standing agenda item — reviewed at the same recurring cadence as sprint planning or a release-readiness sync, so "can we ship the redesign this week" is answered by a number already on the screen, not a fresh argument. Third, an exception to a freeze needs a named escalation path, not a quiet override — Google's own SRE practice is explicit that a disputed freeze goes to an agreed-upon senior executive, logged, rather than getting relitigated informally by whoever pushes hardest; see Google & the birth of error budgets for where that discipline came from.

Team topology changes what "the budget" even refers to. A stream-aligned team's error budget covers their service's SLO — checkout, search, whatever they own. But if you've stood up a platform team per the section above, that team's golden path — the shared CI system, the self-service provisioning API — needs its own SLO and its own budget, because an outage in the paved road burns every downstream team's velocity simultaneously, not just one team's. Conflating the two is a common mistake: a platform team that's never been asked to define its own reliability target has no principled way to say no to "just one more feature" the week its own provisioning API is flaking. Treat the platform the way monitoring & observability teaches you to treat any tier-1 service — with its own golden signals and its own budget, watched the same way.

# checkout-service — error-budget policy
# Owners: Checkout stream-aligned team (eng) + Checkout PM (product)
# Reviewed: every release-readiness sync (weekly, Tuesdays)

SLO:    99.9% successful checkout requests, rolling 30 days
Budget: 43.2 minutes / month

> 75% remaining    -> ship on the normal cadence
25-75% remaining   -> ship, but every release gets a canary step
< 25% remaining    -> feature freeze; only changes that reduce the
                       error rate deploy
Exhausted          -> hard freeze; an exception requires sign-off
                       from [VP Engineering] AND [Head of Product],
                       logged in #release-exceptions with a reason
                       and an expiry date

Any incident that burns more than 10% of a month's budget in one
event gets a blameless postmortem within 5 business days, and at
least one resulting action item lands in this team's sprint backlog
before the freeze lifts.

That last line is the whole page in miniature: the org shape decides who owns the number, the postmortem is what turns a budget-burning incident into an actual fix instead of a recurring one, and the budget itself is what decides, honestly, when the team gets to ship again.

The operating-model scorecard

☺ Like you're 10: A short report card — score each row 0, 1, or 2, and the lowest rows tell you exactly what to fix first.

Score your own team against these seven rows, 0 (not really), 1 (partly), or 2 (solidly true), for a total out of 14. The lowest-scoring rows are next quarter's actual roadmap, not a wish list.

PracticeWhy it mattersHow to tell you're doing it
No gatekeeper bottleneckA single approval queue can't scale with headcount and recreates the wall of confusion.A change's approval time doesn't depend on how many other teams are also shipping this week.
Platform team is pulled, not pushedCapability built ahead of demand is cognitive load nobody asked for.You can name the team that asked for each capability and the toil it removed.
Escape hatches are realA paved path with no exit traps the 20% of legitimate edge cases.A team has actually stepped off the golden path this year and stayed supported.
Postmortems produce closed action itemsAn action item that never closes is the same failure, scheduled for later.You can pull up last quarter's postmortems and see items marked done, not just filed.
Postmortems are read outside the teamSharing is what stops the same failure recurring in a different service.Someone from another team can name an incident they only learned about from a postmortem.
Error-budget policy is written and jointly ownedA policy negotiated during the incident just becomes the argument it was meant to end.Both engineering and product signed the policy before the last time it mattered.
The freeze has actually happened at least onceA policy that's never been enforced is decorative, not operating.You can name the date of the last real feature freeze and what it fixed.
✎ Try it

Pick your own team's lowest-scoring row above. Write one sentence naming the specific evidence that would move it from a 0 or 1 to a 2 — not a feeling, an artifact: a signed policy document, a closed action item, a named team that asked for a capability. If you can't name the evidence, the practice probably isn't real yet, no matter how confident the score felt a minute ago.

🎬 At the Ship-It Guild
👺

Gizmo: Reliability's been shaky lately. Simple fix — stand up a Central Release Board. Nothing ships without their sign-off. Very responsible! 🤑

🦉

Professor Owl: That board has none of the context the team that wrote the change has, Gizmo. It can't judge risk — it can only make everyone wait the same amount, always.

🦊

Foxy: And it puts Ops back in charge of approving Dev's work. That's the wall of confusion with a nicer name on the door.

🐢

Timmy the Turtle: The actual lever's the error budget. Healthy budget, ship freely. Thin budget, slow down. It's a number, not a vote.

🦫

Benny the Beaver: And when something does break, I want the postmortem asking how the config reached prod without a canary — not who typed it.

👺

Gizmo: Fine, fine. No board. I'll go find my shortcut somewhere with worse guardrails.

✓ Checkpoint

1. What is a "gatekeeper team," and why does it fail for a structural reason rather than just a people reason? 2. Name three of the seven mechanics that keep a blameless postmortem genuinely blameless rather than blame with extra paperwork. 3. Why does a platform team need its own SLO and error budget, separate from any single stream-aligned team's? 4. What three habits turn an error budget from a formula into a standing operating ritual? 5. What's the single most reliable way to tell whether a team's "blameless" process is real or just theater?

Check your answers
  1. A gatekeeper team is a single, centralized team (Release Engineering, Change Management, a CAB) that must approve every production change regardless of its actual risk. It fails structurally because the gate has none of the context the team that wrote the change has, so it can't assess risk — it can only make everyone wait the same amount, and that queue gets slower exactly as the organization grows.
  2. Any three of: a facilitator who wasn't the incident commander; a timeline built from logs and chat transcripts, not memory; questions phrased as "how," never "who"; naming multiple contributing factors instead of one root cause; watching who's in the room and what authority they hold over the people speaking; action items with an owner, due date, and a home in the real backlog; publishing the postmortem org-wide.
  3. Because an outage in the platform's shared paved path — the CI system, the provisioning API — burns every downstream team's velocity simultaneously, not just one team's. Without its own SLO, the platform team has no principled way to say no to new feature work the week its own reliability is degraded.
  4. A written policy jointly signed by engineering and whoever owns the roadmap, agreed on while the budget is healthy; a standing review of budget status at a recurring cadence rather than an ad hoc check; and a named escalation path for exceptions (e.g. a specific executive, logged) instead of a quiet informal override.
  5. Pull up several past postmortems and check how many of their action items actually closed. A document with the right section headers and a calm meeting can still be theater if nothing changes afterward — closed action items are the evidence a feeling can't fake.