Delivery Pipeline · Deployment strategies

Deployment strategies

Getting a new build into production is the easy part; getting it there without an outage and with a fast way back out is the actual engineering problem. This page covers the four deployment patterns that answer that problem — rolling, blue-green, canary, and feature flags — how each one shifts traffic, what it costs, and how to design the rollback and the health checks that every one of them depends on to be safe.

Rolling deployment

A rolling deployment replaces instances of the old version with the new version incrementally, a batch at a time, against a single running environment. A typical Kubernetes Deployment with the default RollingUpdate strategy might set maxUnavailable: 25% and maxSurge: 25%: it brings up new pods, waits for each to pass its readiness check, then terminates an equivalent number of old pods, repeating until the whole fleet is on the new version. No second environment is provisioned, so it is the cheapest strategy in raw infrastructure cost — you never pay for double capacity.

The cost shows up elsewhere. For the duration of the rollout, old and new versions serve traffic side by side, which means your API and data layer must tolerate two versions of the application talking to the same database schema and calling each other's endpoints simultaneously — a real constraint on schema migrations, covered from the pipeline side in CI/CD pipelines. Rollback is also slower than the alternatives below: undoing a rolling deployment means running another rolling deployment, in reverse, instance by instance, which takes real time and briefly reintroduces the same mixed-version window in the other direction.

Rolling is the right default for internal services, stateless APIs, and any team without spare capacity budget for a full duplicate environment. It becomes the wrong choice when a bug needs to be off within seconds — a slow rollback under active user impact is the scenario the next strategy is built to avoid.

Blue-green deployment

Blue-green deployment keeps two complete, independent production environments — call them blue (currently live) and green (idle). A new release deploys entirely into the idle environment, where it can be smoke-tested against production-grade infrastructure and real backing services without receiving any live user traffic. Once green passes verification, a router — a load balancer target-group swap, a DNS cutover, or a service-mesh routing rule — redirects 100% of traffic from blue to green in one atomic switch. Blue stays fully provisioned and idle for a defined soak period; if green misbehaves, rollback is the same switch run backward, typically seconds, not a redeploy.

That speed is the whole value proposition, and it costs exactly what it sounds like it costs: two full production-equivalent environments running at once, at least during the cutover window. For a fleet of any size that's a real budget line, which is why blue-green is most common where downtime is unacceptable and the environment is small or mid-sized enough that doubling it briefly is affordable — payment processing, checkout, auth. It also does not, by itself, limit blast radius the way canary does below: the cutover is instant and total, so if a defect wasn't caught in green's pre-cutover testing, 100% of users hit it the moment the switch flips, not a controlled sample.

Stateful systems complicate blue-green further than the diagram suggests: if blue and green share one database, a schema change has to be backward- and forward-compatible with both versions of the application code for the entire soak window, the same constraint rolling deployments live with.

Canary releases and how they differ from A/B testing

A canary release sends a small percentage of real production traffic — 1%, 5%, 10% — to the new version while the rest keeps hitting the old one, then expands that percentage in steps as long as monitored metrics (error rate, p99 latency, saturation — the golden signals covered in monitoring & observability) stay within an agreed threshold. If a step regresses those metrics, traffic routes back to zero on the new version automatically or by a human call, and only a small, bounded fraction of users were ever exposed. Tools like Argo Rollouts, Flagger, and most managed load balancers' weighted-target-group routing implement this natively, often coupled to an automated analysis step that promotes or aborts the rollout based on a Prometheus query rather than a person watching a dashboard.

It is easy to conflate a canary with an A/B test because both route a percentage of traffic to a variant, but they answer different questions for different audiences. A canary is a reliability signal: does the new build crash less, error less, and perform at least as well as the old one, measured in infrastructure and application metrics, run by the platform/SRE team, torn down the moment the rollout completes because both cohorts end up on the same code. An A/B test is a product or business experiment: does variant B convert better, retain better, or earn more revenue than variant A, measured in business and behavioral metrics with statistical significance testing, run by product or growth, and deliberately kept running for a fixed sample-size window because the split itself is the point, not a step toward eliminating one side. Running both concepts under the label "canary" is a common source of confused rollout decisions: someone aborts a legitimate A/B test because "the canary looks worse" on a business metric it was never designed to protect, or ships a reliability canary to 100% without ever checking whether it wins on the product metric a real experiment would have measured.

Feature flags: decoupling deploy from release

All three strategies above move traffic between versions of already-deployed code. A feature flag (or dark launch) works one level up: it ships the new code path to every instance in production, on every version, and controls whether that path executes at runtime with a config lookup instead of an infrastructure routing rule. As covered in version control & branching, this is what makes trunk-based development safe — incomplete or risky work merges to main and deploys continuously, switched off, and gets released later by flipping a flag rather than shipping a new build.

The practical benefit is that a flag flip is not a deployment at all — no build, no rollout, no new artifact — so it is faster than any of the strategies above at the one thing all of them exist to do: turn a bad change off. It also composes with them rather than replacing them: a canary controls which infrastructure a request lands on; a flag controls which code path executes once it gets there. A mature rollout often uses both — canary the infrastructure to a small percentage while additionally gating the risky logic behind a flag, so a bad change can be killed by flag flip in milliseconds without waiting for the canary's own traffic-shift or rollback mechanics.

The trade-off is code complexity, not infrastructure cost: every live flag is a conditional that both states of the code must support, and a flag left in place after full rollout is dead weight a future engineer has to reason around. Treat flag removal as a required follow-up task, not an optional cleanup.

Rollback design, zero-downtime requirements, and choosing a strategy

None of the strategies above are safe by default — they're safe because of two things designed alongside them: a rollback path that's faster than the incident, and health checks that keep traffic off instances that aren't ready to serve it.

Rollback strategy has to be decided before the deploy, not improvised during an incident. The three questions worth answering in advance: What triggers a rollback — a human judgment call, or an automated policy tied to a specific metric threshold (say, 5xx rate above 1% for two consecutive minutes)? How fast can it execute — blue-green's router swap and a flag flip are both seconds; a rolling deployment's reverse rollout and a database migration's down-migration are both materially slower and need to be budgeted for, not discovered under pressure. And is the rollback actually safe — a schema migration that isn't backward-compatible turns "roll back the app" into "the old app now crashes against the new schema," which is why additive, backward-compatible migrations (expand-then-contract) are the norm for any strategy that runs mixed versions even briefly.

Zero-downtime deployment means users never see a failed request because of the deploy itself, and it depends on two distinct Kubernetes-style health checks (the same pattern applies under any orchestrator, including plain load-balancer health checks): a readiness probe, which asks "should the router send this instance traffic right now," and a liveness probe, which asks "is this process still functioning or does it need to be restarted." A new instance that has started but hasn't finished loading its config, warming a cache, or opening a database connection pool must fail its readiness probe — the router then simply skips it, rather than routing live requests to a process that will error on every one. A liveness probe catches a different failure: a process that's technically running but permanently wedged (deadlocked, out of file descriptors), and its failure triggers a restart rather than a traffic reroute. Confusing the two is a common misconfiguration: pointing liveness at a check that depends on a downstream dependency (like a database) causes healthy application processes to get restarted in a loop every time that dependency has a blip, which is exactly the kind of self-inflicted outage a correct readiness/liveness split is meant to prevent.

Choosing among the strategies above is rarely purely technical — it's a function of blast-radius tolerance, budget, and how observable your metrics are. Rolling deployment fits low-cost, low-drama services where a brief mixed-version window is acceptable. Blue-green fits services where downtime itself is the risk to eliminate and doubling infrastructure briefly is affordable. Canary fits services with strong, real-time metrics and a team willing to watch (or automate watching) a gradual rollout, and it's the strategy of choice when the blast radius of a bad deploy needs to be capped, not just recovered from quickly. Feature flags are less an alternative to the other three than a layer that makes all of them safer, by separating the infrastructure risk of deploying from the product risk of releasing. Most mature platforms run some combination: rolling or canary at the infrastructure layer, flags at the application layer, all of it wired into the pipeline described in CI/CD pipelines. Every strategy above is ultimately a statement about how much of production is exposed to an unproven change at once, and how quickly that exposure can be reversed — pick based on which failure mode, a slow rollback or a large blast radius, your service can least afford.

Check your answers

1. Because the rollout happens instance by instance against one environment, old and new versions serve traffic simultaneously for the duration of the rollout, so both must work against the same schema and each other's APIs. Rollback is slower because undoing it means running another rolling deployment in reverse, instance by instance, rather than a single switch.
2. The router (load balancer, DNS, or service mesh) simply points back at the blue environment, which is still fully provisioned and idle — a single atomic switch. The cost is running two complete production environments simultaneously, at least through the cutover and soak window.
3. A canary is a reliability signal — does the new build error and perform as well as the old one — measured in infrastructure metrics by the platform/SRE team and torn down once the rollout completes. An A/B test is a product/business experiment — does variant B convert or retain better — measured in business metrics with statistical significance, run by product/growth, and kept running for a fixed sample window because the split is the point.
4. A readiness probe determines whether the router should send an instance traffic right now (a not-ready instance is skipped, not restarted); a liveness probe determines whether the process is still functioning and should be restarted if not. Pointing the liveness check at something that depends on a downstream service (like a database) means a normal blip in that dependency looks like the application itself is wedged, so a perfectly healthy process gets killed and restarted repeatedly.