Case study: a 90-day transformation
Northwind Retail is a fictional mid-size e-commerce company: about 40 engineers, a Postgres-backed catalog and checkout system, and roughly $80M in annual revenue moving through it. In Q1 2026 their VP of Engineering committed to a 90-day plan to fix a release process that was actively costing the company money. What follows is that plan, quarter by quarter, using the same practices covered earlier in this course — not as abstractions, but as the specific decisions one team made and what happened after.
Imagine a school that only hands back graded tests once every three months, and every time they do, half the class finds out they misunderstood the material weeks ago and now has to relearn it under pressure. That's Northwind at the start of this story: work piled up for months, then got dumped into production all at once, and nobody found out what broke until it was already live. The fix wasn't "work harder" — it was handing tests back constantly, in small pieces, and catching mistakes the day they happened instead of the day of the exam.
The starting point: quarterly releases and Friday-night fire drills
At the start of the year, Northwind shipped to production once per quarter. Feature branches lived for six to ten weeks before merging, so every release was really a merge of several teams' work happening for the first time, under a deadline, days before launch. There was no automated test suite in the deploy path — QA was a manual regression pass run by two people over three days, and it caught what it caught. Dev and Ops sat in separate reporting lines with separate goals: Dev was measured on features shipped, Ops was measured on uptime, and the release itself was the boundary where those incentives collided.
Releases went out Friday evening, on the theory that a quiet weekend gave the team room to fix anything that broke without customer traffic watching. In practice this meant the on-call rotation dreaded every release night, and it meant problems discovered Saturday morning sat half-fixed until Monday because the people who understood the change were asleep or offline. Northwind averaged 1.7 production incidents per release, with a mean time to restore service north of two days — long enough that a bad Friday routinely became a bad week.
Days 1–30: trunk-based development and feature flags
Phase one didn't touch infrastructure at all — it targeted how code moved from a developer's machine into the shared codebase. The team's existing workflow relied on long-lived feature branches, covered on version control & branching as one of the clearest predictors of integration pain: the longer a branch lives apart from trunk, the more it diverges, and the more its eventual merge looks like a small war. Northwind set a hard rule — no branch older than two days — and paired it with feature flags so that half-finished work could merge to trunk constantly without being visible to real users.
This is the trunk-based development model discussed on what is DevOps: small, frequent integrations instead of rare, large ones. The catalog team was first to convert, wrapping their in-progress search-reranking work behind a flag and merging daily instead of holding a branch for three weeks. Merge conflicts, previously a half-day tax on nearly every release, dropped to a handful of lines resolved in minutes because no two engineers' changes were ever more than a day apart. By day 30, four of six product teams had adopted the flag-and-trunk pattern; the other two were mid-migration.
Days 31–60: a real pipeline, with tests and canary deploys
Trunk-based development only pays off if what lands on trunk is actually verified before it ships, so phase two built the pipeline covered on CI/CD pipelines. Northwind stood up a build stage that ran on every commit — unit tests, then an integration suite against a seeded staging database, then a contract test against the payments API — with a hard gate: a red pipeline could not merge to trunk, no exceptions, no "I'll fix it in the next PR." Getting the test suite fast enough to not become its own bottleneck took real work; the team parallelized test shards across runners to hold the full pipeline under nine minutes.
On the deploy side, Northwind replaced the quarterly big-bang release with the canary pattern from deployment strategies: a new build first took 5% of production traffic, held there for 15 minutes while error rate and latency were checked against the previous version, then stepped to 25%, 50%, and 100% if those checks stayed clean — with an automatic rollback to the prior version if any step failed its threshold. Combined with the trunk-based flow from phase one, this meant a single commit could go from merged to fully live in under two hours, instead of waiting for the next quarterly window. The team ran its first mid-week, mid-day deploy in this phase — something that would have been unthinkable three months earlier.
Days 61–90: infrastructure as code and blameless operations
The last phase addressed the two things still done by hand: environment setup and incident response. Northwind's staging environment had drifted from production for over a year — different Postgres extension versions, a missing Redis instance, environment variables set manually by whoever provisioned the box. Following infrastructure as code, the team wrote Terraform definitions for the full environment — network, database, cache, and application tier — and rebuilt staging from that definition rather than patching the existing one. Reviewers could now see infrastructure changes in a pull request diff instead of finding out about them after something broke.
On the operations side, Northwind adopted the golden-signal dashboards from monitoring & observability — latency, traffic, errors, and saturation for checkout and catalog — replacing a scattered set of per-service graphs nobody checked proactively. Every incident, starting in this phase, closed with a blameless postmortem following the pattern in incident management: a written timeline, contributing factors, and action items assigned to a system or a process, never to a person. The first postmortem under the new format covered a canary rollback triggered by a Redis connection-pool exhaustion bug — caught at 5% traffic instead of 100%, and fixed within the same day instead of carrying into the following week.
Northwind's 90 days weren't three unrelated initiatives — they were sequenced so each phase made the next one possible. Trunk-based development produced small, reviewable diffs; small diffs made an automated test suite fast enough to run on every commit without becoming a bottleneck; a fast, trustworthy pipeline made canary deploys viable, because a bad canary could be traced to a small, recent change instead of a quarter's worth of tangled commits. Infrastructure as code and blameless postmortems then closed the loop by making the environment itself reproducible and making failures a source of process fixes instead of blame. Skipping straight to canary deploys without trunk-based development first would have meant canarying enormous, hard-to-diagnose batches — the practices depend on each other in this order.
Before and after, in DORA terms
Using the four metrics from measuring success: the DORA metrics, Northwind's numbers moved from solidly Low tier to solidly High tier over the 90 days — not Elite, and the team was explicit internally that Elite (multiple deploys a day, sub-hour lead time) was a further-out goal, not a 90-day one.
| Metric | Before (Day 0) | After (Day 90) |
|---|---|---|
| Deployment frequency | Quarterly (~4/year) | Several times per week |
| Lead time for changes | 6–10 weeks | Under 2 days |
| Change failure rate | ~35% | ~12% |
| Time to restore service | ~52 hours | ~90 minutes |
The change failure rate improvement is the one worth sitting with: Northwind deployed far more often after the 90 days, yet a smaller share of those deploys caused a customer-facing problem. That's the throughput-and-stability pattern from the DORA research holding true on a real team — canary deploys catching bad changes at 5% traffic instead of 100%, and a fast test suite catching most defects before they reached production at all, rather than more frequent deploys simply creating more frequent breakage.
None of this came from a single tool purchase or a reorg announcement. The 90 days worked because each phase was a change to how work actually moved through the system — branch lifetime, test gating, deploy shape, environment definition, incident follow-up — and each one was kept in place while the next was added. A team that adopted canary deploys but kept six-week feature branches would still be canarying enormous, hard-to-diagnose batches; the sequencing mattered as much as any individual practice.
1. Why did Northwind tackle trunk-based development and feature flags before building out the CI/CD pipeline, rather than the other way around? 2. What specifically changed about how Northwind ran postmortems in phase three, and what principle does that follow? 3. Northwind's change failure rate went down even as deployment frequency went up — why does that match DORA's research rather than contradict it? 4. What kept Northwind at High tier rather than Elite tier at the end of the 90 days?
Check your answers
- Because small, frequent, trunk-based commits are what makes an automated test suite and later a canary deploy pipeline practical and fast to diagnose — verifying and progressively rolling out large, infrequent, long-lived branches would have meant tracing failures back through weeks of tangled changes instead of a single recent commit.
- Postmortems became blameless: a written timeline and contributing factors with action items assigned to systems or processes rather than to individuals, following the pattern from incident management. This follows the principle that reliable failure analysis requires people to report what actually happened without fear of personal blame.
- DORA's research finds throughput and stability move together rather than trading off, because both are downstream of the same practices — small batch sizes, automated testing, and progressive rollout. Northwind's canary deploys caught problems at 5% traffic and its fast test suite caught defects before production, so shipping more often didn't mean breaking more often.
- Elite tier requires on-demand deploys (multiple per day) and sub-hour lead time; Northwind reached several deploys per week and under-two-day lead time — real progress, but the team itself treated further gains toward daily-or-faster deploys and hour-scale lead time as a goal beyond the 90-day window, not something the plan claimed to achieve.