Flagger

Flagger is a Kubernetes operator that turns a release into an experiment the platform runs for you: you point a Canary resource at an ordinary Deployment, and from then on every image change is rolled out to a sliver of real traffic, judged against real metrics every minute, and either promoted step by step to 100% or rolled back automatically — with no human watching a dashboard and no pipeline holding a cluster credential. It solves the platform problem that “deploy” and “release” must stop being the same terrifying moment.

What Flagger is and the problem it solves

☺ Like you’re 10: A helper that lives in your cluster, tries the new version on a few people first, and undoes it if they don’t like it.

Flagger is a progressive-delivery operator from the Flux family, donated to the CNCF and maintained alongside Flux CD. It is not a pipeline runner, not a deployment tool, and not a replacement for GitOps — it sits downstream of all three. Once Argo CD or Flux has reconciled a new image tag into the cluster, Flagger notices the change to the pod template and takes over the last, riskiest hundred metres: getting that new binary in front of users without hurting them.

The problem: “it deployed fine” is not “it works”

A plain rolling update is optimistic. A readiness probe only knows whether a process answered a health check — not whether checkout now returns 500s for German customers, or whether p99 latency tripled because someone dropped an index. By the time a human notices on a dashboard, 100% of traffic has been on the bad version for minutes. The DORA metric this destroys is change failure rate, and the human cost is a 2am page.

The fix, as the CI/CD & Progressive Delivery lesson sets out, is to separate deploy from release and gate the second on evidence. Doing that by hand — shift weight, query Prometheus, decide, un-shift — is exactly the repetitive judgement work a platform should absorb. Flagger absorbs it.

The Flagger bargain: keep your Deployment

Flagger’s defining design choice is that it wraps your workload rather than replacing it. Your Deployment stays a Deployment; your Helm chart, Kustomize base and CI image bump are untouched. You add one resource — a Canary with a targetRef — and Flagger builds the machinery around it. That is why Flagger retrofits cheaply onto a fleet of existing services: the adoption diff is one new YAML file per app, not a rewrite of every workload manifest.

What Flagger does not do

Flagger does not build images, does not run tests (it calls things that do, via webhooks), and — critically — does not route traffic itself. It writes routing configuration into whatever traffic layer you already run: an Istio VirtualService, an SMI TrafficSplit for Linkerd, an NGINX Ingress carrying nginx.ingress.kubernetes.io/canary annotations, a Gateway API HTTPRoute. No mesh and no supported ingress means no weighted split, which rules out percentage canaries (blue/green is still possible; see below).

One version detail worth carrying into an interview: the Linkerd provider still drives SMI TrafficSplit objects, and from Linkerd 2.12 onwards SMI is no longer built in — you must install the SMI extension separately or Flagger has nothing to write weights into. Check what your traffic layer actually exposes before assuming the provider works.

Where Flagger fits in a platform

☺ Like you’re 10: Flagger stands at the very end of the delivery conveyor belt, right where the toys reach the children.

Which plane it serves

In the platform architecture layering, Flagger is a control-plane operator serving the delivery domain. It runs as one Deployment per cluster (often in the mesh’s namespace), watches Canary resources across namespaces, and drives two data planes it does not own: the traffic layer (mesh or ingress) and the metrics layer (Prometheus). That triangle — operator, router, metrics store — is the whole architecture.

Its neighbours

Flagger is only as good as the tools either side of it:

• Upstream: CI and a GitOps reconciler put the new image tag in the cluster. Flagger reacts to that; it never pulls from Git itself.
• Sideways: the mesh or ingress controller does the weighted routing, and mesh telemetry supplies the golden signals — see Networking & Service Mesh.
• Underneath: Prometheus stores the metrics Flagger queries — see Observability.
• Around it: the self-service layer. A mature platform templates the Canary from a golden path so “safe release” is the default, not an opt-in.

CNPE domain relevance

Flagger sits in Domain 2 — GitOps & Continuous Delivery (25%), under progressive delivery and safe rollout strategies. It also brushes Domain 4 — Observability (the analysis is only as good as your SLIs) and Domain 3’s operator patterns, because Canary is a textbook operator-pattern CRD: a declarative API encoding an expert workflow that used to live in a runbook.

How Flagger works

☺ Like you’re 10: It makes a twin of your app, gives the twin a little bit of the traffic, checks the scoreboard every minute, and slowly hands over — or takes it all back.

The objects Flagger creates for you

This surprises everyone once. Create a Canary named checkout targeting Deployment checkout, and Flagger generates a family of objects, then scales your original Deployment to zero:

The analysis loop

Flagger watches the spec.template of the target Deployment (plus tracked ConfigMaps and Secrets). When it changes — a new image tag arrives from Git — Flagger declares a new revision and starts an analysis. Every interval it does the same three things: run the metric queries, run the webhooks, then either add stepWeight to the canary’s traffic or record a failure. When failures reach threshold, it aborts: weight to 0%, canary scaled down, alert fired. When weight reaches maxWeight with checks still passing, it promotes — copying the canary’s pod spec onto the primary Deployment, waiting for that rollout, then routing 100% back to primary and scaling the canary to zero.

The CRDs Flagger introduces

All three live under the flagger.app/v1beta1 API group, and knowing which is which is worth exam marks:

• Canary — the main event. Declares the target workload, the service/routing shape, and the analysis policy.
• MetricTemplate — a reusable, parameterised query against a metrics provider (Prometheus, Datadog, New Relic, CloudWatch, Graphite, Dynatrace, InfluxDB…). Referenced from a Canary by templateRef.
• AlertProvider — where to send rollout notifications. spec.type is one of slack, msteams, discord, rocket or gchat; referenced from analysis.alerts[].providerRef.

The resources you will actually write

☺ Like you’re 10: Three little files: the rules for the taste test, a custom thing to measure, and a way to do it without a real crowd.

A canary release on Istio

This is the shape you should be able to produce from memory. Read it top to bottom: what to watch, how it is exposed, and how to judge it.

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: checkout
  namespace: prod
spec:
  provider: istio                     # istio | linkerd | nginx | traefik | contour | gloo | kuma |
                                      # appmesh | skipper | apisix | osm | knative |
                                      # gatewayapi:v1 | gatewayapi:v1beta1 | kubernetes
  targetRef:                          # your ordinary Deployment — you do NOT rewrite it
    apiVersion: apps/v1
    kind: Deployment
    name: checkout
  autoscalerRef:                      # Flagger clones this HPA as checkout-primary, targeting the primary
    apiVersion: autoscaling/v2
    kind: HorizontalPodAutoscaler
    name: checkout
  progressDeadlineSeconds: 600        # canary Deployment must make progress within 10m, else roll back
  service:
    port: 80                          # port on the generated Services
    targetPort: 8080                  # container port
    portDiscovery: true               # also expose other container ports found on the pod
    gateways:
      - istio-system/public-gateway   # Istio Gateways to attach the VirtualService to
    hosts:
      - checkout.acme.com
    trafficPolicy:
      tls: { mode: DISABLE }
    retries:
      attempts: 3
      perTryTimeout: 2s
  analysis:
    interval: 1m                      # run the checks every minute
    threshold: 5                      # 5 failed checks in a row -> abort and roll back
    maxWeight: 50                     # ramp to 50%, then promote straight to 100%
    stepWeight: 10                    # +10% per successful interval: 10,20,30,40,50
    metrics:
      - name: request-success-rate    # BUILT-IN: non-5xx share of requests, from mesh telemetry
        thresholdRange: { min: 99 }   # need >= 99%
        interval: 1m
      - name: request-duration        # BUILT-IN: p99 latency in milliseconds
        thresholdRange: { max: 500 }  # need p99 <= 500ms
        interval: 30s
      - name: checkout-error-budget   # CUSTOM: resolved via the MetricTemplate below
        templateRef:
          name: error-ratio
          namespace: prod
        thresholdRange: { max: 1 }
    webhooks:
      - name: smoke-test
        type: pre-rollout             # runs BEFORE any traffic shifts; failure aborts immediately
        url: http://flagger-loadtester.prod/
        timeout: 30s
        metadata:
          type: bash
          cmd: "curl -sd 'test' http://checkout-canary.prod/health | grep ok"
      - name: load-test
        type: rollout                 # runs at EVERY interval, to generate representative traffic
        url: http://flagger-loadtester.prod/
        timeout: 5s
        metadata:
          type: cmd                   # loadtester command types: cmd (default) | bash | helmv3 | bats
          cmd: "hey -z 1m -q 10 -c 2 http://checkout-canary.prod/"   # port 80, per service.port above
      - name: hold-for-humans
        type: confirm-promotion       # Flagger waits for HTTP 200 here before promoting to 100%
        url: http://release-gate.prod/approve
    alerts:
      - name: on-call
        severity: error               # info | warn | error
        providerRef:
          name: slack-platform
          namespace: flagger-system

Four fields do the heavy lifting and the exam likes all four: interval (how often), threshold (failures before abort), stepWeight (size of each traffic increase), maxWeight (the ceiling before full promotion). With the numbers above, a clean release takes about five minutes — and a broken one is dead in five.

A MetricTemplate for anything the built-ins miss

The built-in request-success-rate and request-duration come from mesh telemetry and cover the two golden signals most releases break. Anything business-specific — cart abandonment, queue depth, a 4xx ratio on one route — needs a MetricTemplate: a reusable PromQL query with Flagger variables ({{ target }}, {{ namespace }}, {{ interval }}) substituted at query time. A template returning no data counts as a failed check — a feature, not a bug.

apiVersion: flagger.app/v1beta1
kind: MetricTemplate
metadata:
  name: error-ratio
  namespace: prod
spec:
  provider:
    type: prometheus
    address: http://prometheus.monitoring:9090
    # secretRef: { name: prom-basic-auth }     # if your Prometheus needs credentials
  query: |
    100 - sum(
      rate(
        http_requests_total{
          kubernetes_namespace="{{ namespace }}",
          kubernetes_pod_name=~"{{ target }}-[0-9a-zA-Z]+(-[0-9a-zA-Z]+)",
          status!~"5.."
        }[{{ interval }}]
      )
    )
    /
    sum(
      rate(
        http_requests_total{
          kubernetes_namespace="{{ namespace }}",
          kubernetes_pod_name=~"{{ target }}-[0-9a-zA-Z]+(-[0-9a-zA-Z]+)"
        }[{{ interval }}]
      )
    ) * 100
---
apiVersion: flagger.app/v1beta1
kind: AlertProvider
metadata:
  name: slack-platform
  namespace: flagger-system
spec:
  type: slack
  channel: platform-releases
  username: flagger
  secretRef:
    name: slack-webhook          # key: address -> https://hooks.slack.com/services/...

Blue/green and A/B — same CRD, different analysis block

You do not learn a second resource for the other strategies — you change the analysis block. Drop maxWeight/stepWeight, add iterations, and you get blue/green: N rounds of checks with no traffic shift, then a 100% flip. Add match rules on headers or cookies alongside iterations and you get A/B testing — a fixed cohort routed to the canary, right when the change is a UI experiment rather than a latency risk. Add mirror: true (Istio only, with an optional mirrorWeight) and production traffic is duplicated to the canary while real responses still come from primary.

# Blue/green WITHOUT a service mesh. The apex Service always selects the primary pods;
# Flagger exercises the new version through the generated checkout-canary Service and
# webhooks, then "flips" by copying the canary pod spec onto the primary Deployment.
spec:
  provider: kubernetes                # no weighted routing available
  analysis:
    interval: 30s
    threshold: 2
    iterations: 10                    # 10 passing rounds, no traffic shift, then flip 100%
    metrics:
      - name: checkout-error-ratio
        templateRef: { name: error-ratio, namespace: prod }
        thresholdRange: { max: 1 }
---
# A/B testing — a cohort chosen by header/cookie, not by percentage
spec:
  provider: istio
  analysis:
    interval: 1m
    threshold: 5
    iterations: 12                    # 12 minutes of exposure to the matched cohort
    match:
      - headers:
          x-canary:
            exact: "insider"
      - headers:
          cookie:
            regex: "^(.*?;)?(beta=always)(;.*)?$"
    sessionAffinity:                  # keep a user on whichever version they first saw
      cookieName: flagger-cookie
      maxAge: 21600

Day-to-day commands

☺ Like you’re 10: There is no special Flagger app to learn — you just ask kubectl how the taste test is going.

Install and wire it up

Flagger installs with Helm — or, GitOps-correctly, as a Flux HelmRelease. Two settings matter: the mesh provider and the metrics server address.

# Install the operator, pointed at Istio and your Prometheus
helm repo add flagger https://flagger.app
helm upgrade -i flagger flagger/flagger \
  --namespace istio-system \
  --set meshProvider=istio \
  --set metricsServer=http://prometheus.monitoring:9090

# Install the load tester in the APP namespace (the webhook target used above)
helm upgrade -i flagger-loadtester flagger/loadtester --namespace prod

# Confirm the CRDs landed
kubectl get crd | grep flagger.app
#   alertproviders.flagger.app  canaries.flagger.app  metrictemplates.flagger.app

Watch, debug, and drive a rollout

# The single most useful command: status and current weight for every canary
kubectl get canaries -A
# NAMESPACE  NAME      STATUS        WEIGHT  LASTTRANSITIONTIME
# prod       checkout  Progressing   30      2026-07-20T09:14:22Z

# Why did it halt? The events on the Canary are the actual answer.
kubectl -n prod describe canary/checkout

# Follow the operator's own reasoning, including every metric value it read
kubectl -n istio-system logs deploy/flagger --tail=200 -f

# Trigger a canary analysis: ANY pod-template change starts one
kubectl -n prod set image deployment/checkout app=ghcr.io/acme/checkout:1.9.0

# Inspect what Flagger generated (note the -primary twins and the zeroed source).
# Do NOT filter on -l app=checkout: the generated primary objects carry app=checkout-primary.
kubectl -n prod get deploy,svc,hpa,virtualservice

# Emergency stop: revert the image and Flagger abandons the run within one interval
kubectl -n prod set image deployment/checkout app=ghcr.io/acme/checkout:1.8.4

# Skip analysis for a hotfix (promote straight through — use sparingly)
kubectl -n prod patch canary/checkout --type=merge -p '{"spec":{"skipAnalysis":true}}'

Notice there is no flagger promote or flagger rollback verb. That is deliberate: promotion is a consequence of evidence, not of a person typing. Manual gating, when you genuinely need it, goes through the confirm-rollout, confirm-traffic-increase and confirm-promotion webhooks — an endpoint that returns 200 to proceed and anything else to hold. Add these to your command reference muscle memory.

Reading the status field

The STATUS column is status.phase, and knowing the vocabulary turns “it’s stuck” into a diagnosis in one glance. The CRD defines these phases:

Two escape hatches are worth memorising alongside the phases. spec.suspend: true pauses all reconciliation for a Canary without deleting it — the right lever during an incident when you want Flagger to stop making decisions but do not want to tear down the primary. And analysis.stepWeightPromotion ramps traffic back to primary gradually during the promotion phase, rather than in one jump, which matters when the primary needs time to scale up under load.

Gotchas and failure modes

☺ Like you’re 10: Four things trip people up: no traffic, no mesh, a robot fighting another robot, and forgetting the twin.

Not enough traffic to judge anything

The number-one Flagger failure. request-success-rate over a minute with three requests is noise; with zero requests it returns no data, Flagger counts a failed check, and after threshold intervals it rolls back a perfectly healthy release. Symptom: the events on the Canary show advancement halted because no values were found for request-success-rate — the wording varies by version, but “halt advancement” plus “no values found for metric” is the signature. Fixes, in order of preference: a rollout webhook driving the load tester so every interval sees representative traffic; a longer interval; or, for genuinely low-traffic internal services, blue/green with iterations instead of a percentage canary. See Triage: Delivery.

No mesh, no weighted split

Flagger cannot invent traffic control. Without Istio, Linkerd, App Mesh, NGINX, Traefik, Contour, Gloo, Kuma, or a Gateway API implementation, a percentage canary is impossible — replica-ratio “canaries” on a plain Service are connection-sticky approximations, not request-level splits, as Networking explains. Set provider: kubernetes and use blue/green, or install a mesh first. Choosing Flagger before choosing a traffic layer is cart-before-horse.

GitOps and Flagger fighting over the same field

Flagger mutates the target Deployment (scaling it to zero) and creates objects your reconciler did not author. An Argo CD Application with selfHeal: true scales the Deployment back up; Flagger scales it down; the loop never settles and the canary never runs. Tell the reconciler not to own that field — Argo CD ignoreDifferences on /spec/replicas (plus argocd.argoproj.io/compare-options: IgnoreExtraneous on generated objects), or the equivalent Flux exclusion. Two controllers claiming one field is a general anti-pattern, not a Flagger quirk.

The quiet ones

• The HPA. Forget autoscalerRef and your HPA keeps targeting the zeroed source Deployment while -primary serves all traffic unscaled. A silent capacity trap.
• Config drift. Flagger copies referenced ConfigMaps and Secrets to -primary versions and treats a change to them as a new revision. To opt one out, annotate that ConfigMap or Secret — not the Deployment or the Canary — with flagger.app/config-tracking: disabled. Getting the annotation onto the wrong object is a classic silent no-op.
• Deleting the Canary. By default this leaves the generated primary objects behind; set spec.revertOnDeletion: true to have Flagger hand the Deployment back cleanly.
• Prometheus scope. The built-ins assume Prometheus scrapes mesh telemetry (Istio’s istio_requests_total, Linkerd Viz). Point metricsServer at one that doesn’t and every check fails identically — cross-check with the troubleshooting playbook.

Alternatives and when to choose it

☺ Like you’re 10: Several tools do the taste test. They mostly differ in how much of your app they ask you to change.

The field

The honest decision rule

Choose Flagger when the mesh is already there and you want progressive delivery to spread across a large estate cheaply — one small file per service, convention over configuration. Choose Argo Rollouts when release engineers want to author the rollout shape, hold it at a step, and press a button. Choose neither, yet, if you have no meaningful SLIs: as the delivery lesson puts it, a canary without metric analysis is just a slow bad deploy. Get observability right first — Flagger is the payoff, not the prerequisite.

Exam relevance and going further

☺ Like you’re 10: Flagger is on the exam’s tool list — but its own website is locked during the test, so the shapes have to already be in your head.

What to know cold

Flagger is on the official CNPE tool list, in the 25%-weighted GitOps & Continuous Delivery domain. Be able to, without notes:

• State in one sentence how Flagger differs from Argo Rollouts (Canary wraps a Deployment; Rollout replaces it) — and never paste one tool’s example into the other, because it will apply cleanly and do nothing.
• Write a minimal Canary: targetRef, service.port, and an analysis with interval, threshold, maxWeight, stepWeight, and one metric with a thresholdRange.
• Name the two built-in metrics (request-success-rate, request-duration) and say where a custom one comes from (MetricTemplate → Prometheus).
• Explain what Flagger generates (-primary Deployment/Service, -canary Service, the routing object) and why the source Deployment shows zero replicas.
• Diagnose a stuck rollout with kubectl get canary -A and kubectl describe canary, and recognise “no values found for metric” as the low-traffic failure.
• Say why a mesh or supported ingress is a prerequisite for percentage-based canaries.

The documentation allowlist — read this twice

⚖ CNPA vs CNPE — That allowlist is a CNPE-specific mechanic — it exists only because CNPE is hands-on and a narrow set of live lookups is technically reachable mid-task. CNPA is stricter, not looser: a fully closed-book, multiple-choice exam with zero external resources and zero lookups of any kind, so there is no Quick Reference box or kubectl explain to fall back on there either. Even so, the concepts above — what a Canary wraps versus what it replaces, the built-in metrics, why the source Deployment reads 0/0 — are exactly the kind of concept-level knowledge CNPA’s closed-book recall draws on.

The practical implication: budget a study session for hand-writing a Canary on a blank page, checking it against Know Cold, and repeating until the field order comes out automatically. Then rehearse the triage half with Triage: Delivery. Reading about Flagger is not the same as recalling it under a timer.

Official links (for study time, not exam time)

• fluxcd.io/flagger — the canonical documentation, including the full Canary spec reference and per-provider tutorials.
• github.com/fluxcd/flagger — source, release notes, and the podinfo demo used in most tutorials.
• Metrics & MetricTemplates and Webhooks — the two pages you will actually re-read on the job.
• Related on this site: CI/CD & Progressive Delivery (the parent lesson), Release Engineering, Glossary, and the practice set in Practice: GitOps.