Flux — Image Automation
Image automation is the pair of optional Flux controllers that close the last manual gap in a delivery pipeline. image-reflector-controller watches a container registry and catalogues the tags it finds; image-automation-controller works out which tag wins and commits it back to Git. Nothing is patched in the live cluster — the bump arrives as an ordinary commit, the ordinary reconcile loop picks it up, and Git stays the source of truth. Argo CD has no first-party equivalent, which makes this the capability platform teams most often name when they choose Flux.
Imagine a robot who stands in the toy factory’s warehouse reading the labels on the shelf. Every few minutes he writes down every box number he can see, and you gave him one rule — “always pick the biggest number” — so he knows which box is newest. Here is the clever part. He does not sneak into your bedroom and swap your toy while you sleep. He walks over to your shopping list, crosses out the old box number, writes the new one, and signs his name. Then the usual helper reads the list and fetches the toy, exactly like always. Because the robot only ever edits the list, the list and the bedroom always agree — and if he picks a bad box, you just cross it out again.
This page is one of the Flux family. The parent page covers the GitOps Toolkit as a whole — sources, Kustomization, the reconcile loop, the flux CLI — and you should read it first if any of that is new. Its siblings cover Helm delivery and multi-tenancy and repository structure. This page owns the two image controllers.
What Flux image automation is and the problem it solves
☺ Like you’re 10: Someone still has to type the new version number into Git. This is the robot that types it for you.
Follow a change through a well-built pipeline and you find one stubbornly human step in the middle. CI builds an image, tags it, signs it, pushes it — automatic. Flux pulls manifests from Git and reconciles them — automatic. Between those halves sits a person opening deployment.yaml and editing the string after the colon. That edit is boring, done under time pressure, and is where somebody fat-fingers v1.2.3 as v1.23. Image automation deletes it: you declare which registry to watch, what “newest” means for your tagging scheme, and which line in which file to rewrite, and the controllers do the typing from then on.
Two controllers, three custom resources
The work splits across two independent Deployments, as the rest of the GitOps Toolkit does. One knows about registries and nothing about Git; the other knows about Git and nothing about registries. They meet through a Kubernetes object with a status field, never a function call.
| Controller | Its one job | CRDs it owns | Talks to |
|---|---|---|---|
image-reflector-controller | List the tags in a registry repository, and apply a rule to decide which is “latest” | ImageRepository, ImagePolicy | The container registry (read-only) |
image-automation-controller | Find the marked-up lines in Git, write the winning tag into them, commit and push | ImageUpdateAutomation | The Git remote (read and write) |
Read that table as the troubleshooting map, because under pressure the reverse lookup is what you need. A tag that should have been picked up but wasn’t is an ImagePolicy problem, so read image-reflector-controller. A tag that was picked but never landed in Git is an ImageUpdateAutomation problem. A commit that is in Git but hasn’t reached the cluster is not an image-automation problem at all — that is the ordinary source-and-Kustomization chain on the parent page.
The pipeline is a ring, not a line. CI pushes an image to a registry; the reflector reads the registry; the automation writes to Git; source-controller reads Git; kustomize-controller writes to the cluster. Every arrow crosses a boundary that is observable and revertible, and no arrow ever points from outside the cluster into the Kubernetes API.
Why the write-back goes to Git, and not to the cluster
There are three ways to get a new image version running. Make the tag mutable — push :latest over the top, set imagePullPolicy: Always, restart the pods. Let a controller inside the cluster mutate the live Deployment when it spots a new tag. Or rewrite the manifest in Git and let the normal reconcile loop do what it always does.
Only the third preserves the invariant that makes GitOps worth the effort. Under the first two the cluster runs something no commit describes, so “what is deployed in prod?” goes back to being answered by looking at the cluster — the pre-GitOps answer the discipline exists to abolish. The four OpenGitOps principles quietly break: desired state stops being declarative and versioned, and the drift-correction loop has nothing to correct toward, because the cluster is now ahead of the repo rather than behind it. Under Flux’s design the new tag is a commit with an author, a timestamp and a diff: git revert is a working rollback, git log -- apps/checkout/ is a working deployment history, a reviewer can be required before it merges, and the reconciler remains the only thing writing to the cluster.
It is the anti-pattern image automation replaces, and it fails in ways that are hard to debug. Two replicas of the same Deployment can end up running two different builds, because they pulled at different moments. A node that already has the tag cached may not re-pull at all. Nothing records which build is live, so an incident timeline has a hole exactly where you need one. And a rollback has nowhere to roll back to, because the previous artefact no longer has a name. Catalogued in Anti-Patterns.
What it deliberately does not do
Image automation chooses a tag. It does not decide whether that tag is safe — no signature verification, no CVE scan, no admission policy, no watching error rates afterwards. Those are four other tools’ jobs (Cosign, Trivy, Kyverno, Flagger), and their staying separate is a feature: a tag-picker that also enforced policy would be a policy engine you could bypass by not using it.
Where it fits in a platform
☺ Like you’re 10: It sits in the gap between “the build finished” and “the deploy started”, and it is the only piece that writes to Git instead of reading from it.
On the plane diagram in Platform Architecture, the image controllers sit at the seam between the build and delivery planes. They are the only part of a Flux installation that pushes to your Git remote rather than pulling from it, and that asymmetry drives most of the decisions on this page: the credentials it needs, the branch strategy you pick, the review gate you keep or give up.
Upstream: what has to be true before this works
Image automation consumes somebody else’s tagging discipline. Whatever builds your images — Tekton, GitHub Actions, Jenkins — must produce tags that are immutable and sortable: immutable so a tag always means one artefact forever, sortable so “newest” has a mechanical answer. If your CI pushes :latest and nothing else, there is nothing for an ImagePolicy to order and no configuration will fix that. Two schemes work well, each mapping to a different policy shape: semantic-version tags cut from releases (1.4.2, 1.5.0-rc.1) for products with a release cadence, and build tags embedding a monotonic component (main-9f2a1c-1727101800, trailing Unix timestamp) for trunk-based teams shipping every merge. The choice is a design decision, treated as one in Release Engineering.
Downstream: the handoff to delivery and progressive rollout
The moment the automation pushes, it is finished and has no further opinions. source-controller notices the revision, the relevant Kustomization or HelmRelease reconciles, the pod template changes. If Flagger watches that workload, a changed pod template is precisely its trigger and canary analysis begins — a complete automatic path from “CI pushed an image” to “10% of traffic is on it and the error rate is being measured”, with a human required at zero points and a commit recording every one of them. CI/CD & Progressive Delivery builds that story up in full. Elsewhere on the road: registry credentials come from a pull Secret or cloud workload identity rather than a pasted token (External Secrets, Secrets Management); the setter markers live inside Kustomize overlays or Helm values in your repo; and if Backstage offers “keep my service on the latest build” as a self-service action, the thing behind that button is an ImagePolicy rendered from a template.
CNPE domain relevance
Image automation lands in Domain 2 — GitOps & Continuous Delivery (25%), the joint-largest slice of the exam blueprint. It appears as a hands-on task (“configure the cluster to deploy new builds of this image automatically”) and as a discrimination question (“which of these approaches keeps Git authoritative?”). Both reward knowing the three CRDs by shape rather than by name. See the Tool Landscape for where Flux sits among the officially named projects.
How it works — the two image controllers
☺ Like you’re 10: One robot reads the shelf and writes a note. The other robot reads the note and edits your list.
Both are ordinary Kubernetes controllers: a Deployment in flux-system, a reconcile loop driven by each object’s interval, a .status with a Ready condition. Neither holds state anywhere except the Kubernetes API and Git. Delete either Deployment and nothing breaks — the cluster stops receiving new tags, and everything already committed keeps reconciling exactly as before. Image automation is additive and safe to switch off, which is why it is reasonable to enable it in dev long before prod.
A plain flux install or flux bootstrap gives you source-, kustomize-, helm- and notification-controller. The image controllers are extra components and must be opted into with --components-extra=image-reflector-controller,image-automation-controller. If your ImageRepository sits there with no status and kubectl get crd | grep image.toolkit comes back empty, this is why — a common first stumble in a timed task. flux check lists the controllers actually running.
image-reflector-controller: scan, catalogue, choose
The reflector does two separable things, which is why it owns two CRDs. An ImageRepository is a scan job: on its interval it authenticates to the registry, calls the tag-listing endpoint for one repository path, and stores the tag names in an internal database inside the controller. It does not pull image layers and does not care what the tags mean. Its status reports when the last scan ran and how many tags came back — the first thing to check when a policy looks stuck, because a policy can only choose from tags the scan actually saw.
An ImagePolicy is a query over one repository’s catalogue: optionally filter the tag list with a regex, optionally extract a substring from each surviving tag, sort what remains by one of three orderings, write the winner into its own status. Several policies can point at the same repository — stable semver for prod, release candidates for staging, main-* build tags for dev. One scan, many answers: the same one-source-many-consumers separation the parent page describes for GitRepository.
image-automation-controller: clone, set, commit, push
The automation controller never watches the registry. Its input is Git plus whatever the policies currently say. On each interval an ImageUpdateAutomation does roughly this:
- Clone (or fetch) the repository named by its
sourceRef, checking out the branch inspec.git.checkout.ref. - Walk every YAML file under
spec.update.path, looking for the setter marker comments described below. - For each marker, look up the named
ImagePolicy, read the image reference it currently selects, and write it into the marked field — but only if it differs from what is already there. - If nothing changed, stop. Do not commit an empty diff.
- Otherwise render the commit message from
messageTemplate, commit as the configured author, and push tospec.git.push.branch(defaulting to the checkout branch). - Record the result in
.status— including the SHA it pushed — and emit an event thatnotification-controllercan forward.
Step 2 is where people come unstuck. The controller edits files as they exist in the repository — a textual set on YAML, not a patch on rendered output. It cannot reach a tag that only appears after a Kustomize build or a Helm render, and it cannot reach anything inside a chart it does not own. Whatever you want automated must be a literal string in a file in your repo, with a comment next to it.
One full lap, with the latencies visible
“It didn’t deploy” is nearly always “it hasn’t deployed yet”. CI pushes ghcr.io/acme/checkout:main-9f2a1c-1727101800. Up to one ImageRepository interval later, the reflector scans and the tag enters the catalogue. On its next tick the ImagePolicy re-sorts. Up to one ImageUpdateAutomation interval later, the automation rewrites the file and pushes. Up to one GitRepository interval later, source-controller fetches the commit. Up to one Kustomization interval later it is applied. Then the Deployment starts rolling. Worst case that is five intervals stacked, so a fifteen-minute lag with five-minute intervals should surprise nobody — and nobody should “fix” it by setting everything to ten seconds, which is how you get rate-limited by your registry and your Git host on the same afternoon. The right answers are modest intervals, a Receiver webhook so the Git half reacts instantly (see the parent page), and flux reconcile when you genuinely cannot wait.
“The thing I had to unlearn is that my merge isn’t the deploy any more. I merge to the app repo, CI builds an image, and then a completely different repo gets a commit from a bot called fluxcdbot about ninety seconds later. Once I understood that the bot’s commit is the deploy, everything made sense — I watch the config repo, not my own. And when something looks stuck I run flux get image policy -A first, because nine times out of ten the policy just hasn’t seen my tag yet.”
The resources you will actually write
☺ Like you’re 10: Three objects and one magic comment. That’s the whole feature.
Everything reduces to: an ImageRepository saying where to look, an ImagePolicy saying how to choose, an ImageUpdateAutomation saying where to write, and a comment saying which line. Learn those four shapes and you can build image automation from a blank file.
Flux promotes its API groups as they stabilise, and the image group has moved: in more recent releases ImageUpdateAutomation has been promoted to a stable v1 while ImageRepository and ImagePolicy have remained at a beta version. The examples below use v1beta2 throughout, but never guess in a task. Run kubectl api-resources --api-group=image.toolkit.fluxcd.io for the exact versions and short names this cluster serves, then kubectl explain imagepolicy.spec --recursive for the real schema. The kinds and field names below are stable; only the version suffix moves.
ImageRepository — scanning the registry
The scan job. Note that spec.image is a repository path with no tag and no digest — putting :latest on the end is rejected, and it is a mistake people make once.
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
name: checkout
namespace: flux-system
spec:
image: ghcr.io/acme/checkout # repository path only — NO tag, NO digest
interval: 5m # how often to re-list the tags
timeout: 60s
secretRef:
name: ghcr-pull # a kubernetes.io/dockerconfigjson Secret
exclusionList: # regexes; tags matching these never enter the catalogue
- '^.*\.sig$' # cosign signature artefacts (excluded by default)
- '^.*\.att$' # attestations, if your build pushes them
suspend: falseThree fields govern authentication and picking the wrong one is a common failure. secretRef points at an ordinary image-pull Secret of type kubernetes.io/dockerconfigjson — the same kind you would put in a Pod’s imagePullSecrets, and the one for Docker Hub, GHCR or a self-hosted Harbor. serviceAccountName names a ServiceAccount in the same namespace whose imagePullSecrets the controller should borrow. And provider switches on contextual login for a cloud registry:
# Elastic Container Registry, using the controller pod's own IAM identity # (IRSA on EKS, or the node role) — no Secret, no rotation, no long-lived token. apiVersion: image.toolkit.fluxcd.io/v1beta2 kind: ImageRepository metadata: name: checkout-ecr namespace: flux-system spec: image: 123456789012.dkr.ecr.eu-west-1.amazonaws.com/checkout interval: 5m provider: aws # generic (default) | aws | azure | gcp
Prefer provider wherever the registry supports it: ECR authorization tokens expire in hours, so the alternative is a CronJob refreshing a Secret forever. On Azure and GCP the equivalent is workload identity bound to the controller’s ServiceAccount. Older Flux releases exposed this as controller-level flags rather than a per-object field, so if provider is missing from the CRD schema, check the controller Deployment’s arguments instead. For a private CA or mutual TLS, certSecretRef supplies the CA bundle and client certificate; insecure: true permits plain HTTP and should never appear outside a local kind cluster.
The default exclusionList already filters out cosign signature tags — the sha256-….sig artefacts that Cosign pushes alongside your image. That default exists because those tags are, technically, tags, and without it an alphabetical policy would happily elect a signature as your newest release. If you push other sidecar artefacts with tag-based schemes, extend the list rather than discovering the problem in prod.
ImagePolicy — choosing the winner
A policy is a sort. Exactly one of three ordering strategies must be set, and which is right is determined entirely by your tagging scheme.
| Ordering | What it does | Use when your tags look like | Watch out for |
|---|---|---|---|
semver | Parses each tag as a semantic version and selects the highest inside a range you specify | 1.4.2, v2.0.0, 1.5.0-rc.1 | Tags that are not valid semver are simply ignored — a typo silently drops out of contention |
alphabetical | Lexicographic sort of the tag (or extracted substring), ascending or descending | Timestamps in a fixed-width sortable format, e.g. 20260817-1432 | Lexicographic is not numeric: "10" sorts before "9" |
numerical | Numeric sort of the tag (or extracted substring), ascending or descending | Build numbers or Unix timestamps, e.g. 1727101800 | Needs the value to parse as a number — almost always requires filterTags with an extract |
The first shape is what you write for a product with releases. Note that order is not a field on semver: semver ordering is defined by the specification and the policy always selects the highest version within the range, so the range does all the constraining.
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
name: checkout-stable
namespace: flux-system
spec:
imageRepositoryRef:
name: checkout # an ImageRepository in this same namespace
policy:
semver:
range: '>=1.0.0 <2.0.0' # stay on the 1.x line; never auto-adopt a major bumpRanges use the usual comparator syntax: >=1.0.0 <2.0.0 pins a major line, 1.4.x pins a minor line, ^1.4.2 means “compatible with 1.4.2”, and >=1.0.0 means “anything from here up” — precisely the range you should not use in production. Pre-release tags such as 1.5.0-rc.1 sort below the corresponding release under semver rules and are excluded from a plain range unless you ask for them, so prod gets pre-release safety for free while a staging policy tracking release candidates needs its range written deliberately.
filterTags — the regex that makes the other two orderings usable
Build tags are rarely sortable on their own. filterTags solves it in two moves: pattern is a regex a tag must match to stay in the running, and extract pulls a substring out of the match to use as the sort key. The distinction that matters, and that exam questions like: the extracted value is what gets sorted, but the original full tag is what gets written into your manifest. You sort on the timestamp; you deploy the whole tag.
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
name: checkout-dev
namespace: flux-system
spec:
imageRepositoryRef:
name: checkout
filterTags:
# Tags look like: main-9f2a1c-1727101800
# ^branch ^sha ^unix timestamp
pattern: '^main-[a-fA-F0-9]+-(?P<ts>[0-9]+)$'
extract: '$ts' # sort on the captured timestamp, not the whole string
policy:
numerical:
order: asc # asc = the LARGEST value winsRead order carefully — it reads backwards to most people. It describes the direction of the sort and the policy takes the last element, so asc selects the greatest value and desc the least. For timestamps and build numbers you almost always want asc; if your automation confidently deploys your very first build and then never moves again, you have set desc.
The pattern is a Go (RE2) regex, so named groups are (?P<name>…) and there are no backreferences or lookaheads. In extract, a group is $name or $1 — and because the parser takes the longest run of letters, digits and underscores as the name, an expression like $ts-suffix binds unexpectedly. Brace it as ${ts} whenever anything follows the reference. Anchor with ^ and $ unless you have a reason not to; an unanchored pattern happily matches a tag from a different pipeline that merely contains your prefix.
Two escaping layers meet here and both bite. YAML: a pattern containing \. or \d must be in single quotes, not double, or the parser eats the backslash before the controller sees it. Regex: . matches any character, so 'v1.2.3' also matches v1x2y3 — write 'v1\.2\.3' when you mean literal dots. When a policy mysteriously matches nothing, print the pattern back out of the live object with kubectl get imagepolicy checkout-dev -o yaml and check it is the string you thought you wrote.
Several policies over one repository is the normal arrangement, and it is how each environment gets a different appetite for risk without scanning the registry three times:
# prod: only tagged releases on the 1.x line
policy: { semver: { range: '>=1.0.0 <2.0.0' } }
# staging: release candidates too
policy: { semver: { range: '>=1.0.0-0 <2.0.0' } }
# dev: every build off main, newest wins
filterTags: { pattern: '^main-[a-fA-F0-9]+-(?P<ts>[0-9]+)$', extract: '${ts}' }
policy: { numerical: { order: asc } }Newer Flux releases also let an ImagePolicy reflect the selected tag’s digest into its status, so a setter can write image:tag@sha256:… and pin the exact artefact rather than a name that could in principle be re-pointed. Worth enabling for anything security-sensitive — check with kubectl explain imagepolicy.spec rather than assuming, since it arrived later than the rest of the API.
The setter markers — the link between a policy and a line of YAML
Nothing so far has touched your manifests. The link is a YAML comment on the same line as the field you want rewritten, containing a small JSON object naming the policy in namespace:name form. Memorise it character-for-character: a marker with a stray space or a smart quote silently does nothing.
spec:
containers:
- name: checkout
image: ghcr.io/acme/checkout:main-9f2a1c-1727101800 # {"$imagepolicy": "flux-system:checkout-dev"}There are three forms, depending on whether the manifest keeps the image reference in one field or splits it:
| Marker | Rewrites | Put it on |
|---|---|---|
# {"$imagepolicy": "ns:policy"} | The whole reference — repository and tag | A container’s image: field |
# {"$imagepolicy": "ns:policy:tag"} | The tag portion only | A standalone tag: field, e.g. Helm values or a Kustomize newTag |
# {"$imagepolicy": "ns:policy:name"} | The repository name portion only | A standalone repository: or newName field |
In a Kustomize overlay you usually mark the images: transformer rather than the Deployment, so the base stays clean and each overlay carries its own policy:
# apps/checkout/overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
images:
- name: ghcr.io/acme/checkout
newName: ghcr.io/acme/checkout # {"$imagepolicy": "flux-system:checkout-dev:name"}
newTag: main-9f2a1c-1727101800 # {"$imagepolicy": "flux-system:checkout-dev:tag"}In a HelmRelease, mark the values in your repo — never expect the automation to reach inside the chart:
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: checkout
namespace: flux-system
spec:
# ... chart, interval, targetNamespace as usual ...
values:
image:
repository: ghcr.io/acme/checkout # {"$imagepolicy": "flux-system:checkout-stable:name"}
tag: 1.4.2 # {"$imagepolicy": "flux-system:checkout-stable:tag"}Four ways to get this wrong, all producing the same symptom — the automation reports success and changes nothing. The marker sits on the line above the field instead of beside it. The JSON uses single quotes or curly “smart” quotes instead of straight double quotes. The value is quoted such that the # becomes part of the string rather than a comment. Or the file is not under spec.update.path, so nothing looked at it. When a marker seems inert, grep for it in the repo the controller actually checked out, not the one on your laptop.
Because the automation resolves marker names against ImagePolicy objects in its own namespace, in practice the namespace in every marker is the automation’s namespace. A marker naming a policy that lives elsewhere is not an error — it simply never matches. This is where the tenancy story starts, and cross-namespace references may be disabled outright on a locked-down installation; see Flux — Multi-Tenancy.
ImageUpdateAutomation — the commit back to Git
The last object ties it together: which repository to write to, which branch to check out, which branch to push to, which path to scan, and what the commit should say.
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
name: checkout-auto
namespace: flux-system
spec:
interval: 5m
sourceRef:
kind: GitRepository
name: platform-config # its secretRef MUST have WRITE access
git:
checkout:
ref:
branch: main # the branch to read
commit:
author:
name: fluxcdbot
email: fluxcdbot@acme.example
messageTemplate: |
chore(deploy): update image tags [skip ci]
Automation: {{ .AutomationObject }}
{{ range .Updated.Images -}}
- {{ . }}
{{ end -}}
push:
branch: main # omit to push back to the checkout branch
update:
path: ./apps/checkout # only files under here are scanned for markers
strategy: Setters # currently the only strategyupdate.strategy: Setters is the only value the field accepts — a deliberate extension point, not a second option you have forgotten. update.path is both a performance control and a blast-radius control: scoping an automation to one application’s directory means it can never rewrite another team’s manifests, and a stray marker elsewhere cannot surprise you. In a repository serving many teams, one narrow automation per team beats one repo-wide automation, and it composes with the RBAC model on the multi-tenancy page.
Commit templating
messageTemplate is a Go text template rendered against a small context: .AutomationObject is the automation’s own name, .Updated.Images ranges over the new image references, .Updated.Files over the files touched, and .Updated.Objects over the Kubernetes objects whose images changed. Recent releases also expose a richer structure describing each individual old-to-new change; check kubectl explain imageupdateautomation.spec.git.commit before relying on it. Take the template seriously — it is the deployment history a future incident responder will read. Name the automation, list the images, and include whatever marker your CI uses to skip builds (commonly [skip ci]). If your organisation requires signed commits, spec.git.commit.signingKey.secretRef points at a Secret holding an OpenPGP private key; without it, a branch protected by a rule rejecting unsigned commits will refuse the push and the automation goes un-Ready.
Push to the same branch, or push to a side branch?
This choice determines whether image automation is “continuous deployment” or “continuous pull-request”, and it is the design question worth thinking hardest about.
| Push to the checkout branch | Push to a separate branch | |
|---|---|---|
| Config | Omit push.branch, or set it equal to the checkout branch | Set push.branch to something else, e.g. flux-image-updates |
| Effect | The new tag is live within one reconcile of the commit | Nothing deploys until a human merges the branch |
| Review | None — the bot is a trusted committer to that branch | Ordinary pull-request review and required checks |
| Suits | Dev and staging; teams shipping every merge to trunk | Production; anything under change-management or audit |
| Cost | Branch protection on that branch has to permit the bot | Someone must actually open and merge the PR |
Be honest about one thing: Flux pushes the branch, it does not universally open the pull request. On GitLab it can pass push options asking the server to create a merge request; on other hosts the usual arrangement is a small CI job or Action watching that branch — plan for that piece rather than discovering a branch nobody has looked at for six weeks. If the side branch drifts far behind the checkout branch the eventual diff can contain surprises, in which case deleting the branch and letting the automation recreate it is the standard cleanup. Most teams converge on same-branch push in dev and side-branch push in prod: graduated trust, expressed in one line of YAML.
Telling somebody it happened
The image objects are event sources like any other Flux object, so an ordinary Alert makes automated deploys visible. Reuse the Provider you already have; only the source kinds are new.
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
name: image-updates
namespace: flux-system
spec:
providerRef: { name: slack }
eventSeverity: info # info, so you get the successful updates too
eventSources:
- kind: ImageUpdateAutomation
name: '*'
- kind: ImagePolicy
name: '*'
- kind: ImageRepository
name: '*'Use info rather than error: a silent successful deploy is exactly what makes people distrust automation, and errors from a failed scan or rejected push arrive through the same subscription. For a dashboard instead of a channel, every one of these objects has a status Grafana can render, as Observability describes.
Day-to-day commands
☺ Like you’re 10: Ask what the robot can see, ask what it chose, and if you are impatient, poke it.
The flux CLI is a thin wrapper over the CRDs — anything below can be done with kubectl, and under exam pressure kubectl get imagerepositories,imagepolicies,imageupdateautomations -A always works even when a flag escapes you.
Installing the controllers and a key that can push
# The image controllers are OPTIONAL components and are not installed by default. # --read-write-key makes bootstrap create a deploy key with WRITE access, which the # automation controller needs in order to push its commits back. Without it the # deploy key is read-only and every push will be rejected. flux bootstrap github \ --owner=acme \ --repository=platform-config \ --branch=main \ --path=./clusters/prod \ --personal \ --read-write-key \ --components-extra=image-reflector-controller,image-automation-controller # Already bootstrapped without them? Re-run bootstrap with the extra flags — # it is idempotent, and it commits the new controller manifests into the same path. # Confirm both controllers are actually running: flux check kubectl -n flux-system get deploy | grep image
A GitHub deploy key created by a plain flux bootstrap is read-only, because reading is all the other four controllers need. The moment you add image automation the same credential is asked to push, and the remote refuses. The symptom is an ImageUpdateAutomation that is not Ready with an authentication or permission error in flux logs, while every other Flux object is perfectly healthy. Fix it by re-bootstrapping with --read-write-key, or by giving the deploy key write access in the Git host’s settings. A protected branch that forbids direct pushes, or requires signed commits or a passing status check, produces the same class of failure for a different reason — check both.
Look: what can it see, and what did it choose?
# Everything image-related, in one table: flux get image all -A # Is the scan healthy? The table shows the last scan time and the tag count. flux get image repository -A # What did each policy actually select? This is the single most useful command # on this page — if the tag you expect is not here, the problem is upstream of Git. flux get image policy -A # Did the automation run, and what did it push? flux get image update -A # The full status, when the table is not enough. Field names in the status have # shifted across releases, so read the whole block rather than a fixed jsonpath. kubectl -n flux-system get imagepolicy checkout-dev -o yaml kubectl -n flux-system describe imagerepository checkout
Nudge, suspend, resume
# Force a registry scan right now instead of waiting out the interval: flux reconcile image repository checkout # Force the automation to evaluate and (if anything changed) commit right now: flux reconcile image update checkout-auto # Stop the robot. Do this BEFORE hand-editing a tag the automation manages, # or it will revert you on its next tick. flux suspend image update checkout-auto flux resume image update checkout-auto # Suspending the scan instead — useful when you are being rate-limited: flux suspend image repository checkout # Controller logs, already filtered to the object you care about: flux logs --kind=ImageUpdateAutomation --name=checkout-auto --follow flux logs --kind=ImagePolicy --name=checkout-dev flux events --for ImageUpdateAutomation/checkout-auto
Generate the YAML rather than typing it
The --export habit is worth more than memorising field names, it is GitOps-correct (a manifest to commit rather than an object applied imperatively), and it is the fastest legitimate route through a timed task.
flux create image repository checkout \ --image=ghcr.io/acme/checkout \ --interval=5m \ --secret-ref=ghcr-pull \ --export > checkout-repo.yaml flux create image policy checkout-stable \ --image-ref=checkout \ --select-semver='>=1.0.0 <2.0.0' \ --export > checkout-policy.yaml # The dev variant: filter to main builds and sort on the captured timestamp. flux create image policy checkout-dev \ --image-ref=checkout \ --filter-regex='^main-[a-fA-F0-9]+-(?P<ts>[0-9]+)$' \ --filter-extract='$ts' \ --select-numeric=asc \ --export > checkout-policy-dev.yaml flux create image update checkout-auto \ --git-repo-ref=platform-config \ --git-repo-path=./apps/checkout \ --checkout-branch=main \ --push-branch=flux-image-updates \ --author-name=fluxcdbot \ --author-email=fluxcdbot@acme.example \ --interval=5m \ --export > checkout-auto.yaml
If a flag will not come to you, flux create image policy --help is on the exam desktop and the documentation site is not. Build the reflex now.
Gotchas and failure modes
☺ Like you’re 10: Nearly every “it didn’t update” is one of five things: the robot can’t see the shelf, the rule matches nothing, the comment is wrong, the key can’t push, or a person is fighting the robot.
Start every investigation the same way, stopping at the first thing that is not what you expect: is the tag in the registry? → did the scan see it? → did the policy pick it? → did the automation commit it? → did the Kustomization apply it? Four of those five are answered by flux get image all -A, which is why it is the first command to type.
| Symptom | Most likely cause | Where to look |
|---|---|---|
The CRDs do not exist at all; flux get image … errors | The two image controllers were never installed — they are opt-in extras | flux check; kubectl get crd | grep image.toolkit |
ImageRepository not Ready, authentication or 401/403 error | Wrong or missing pull Secret; expired cloud token; wrong provider | flux logs --kind=ImageRepository |
| Scan is Ready but the tag count is 0 or far too low | Wrong repository path in spec.image, or an exclusionList that is too greedy | flux get image repository; kubectl describe |
ImagePolicy not Ready — “no image found” / cannot determine latest | filterTags.pattern matches nothing, the range excludes every tag, or the tags are not valid semver for a semver policy | kubectl get imagepolicy … -o yaml |
| The policy picks the oldest tag and never moves | order: desc where you meant asc | The policy spec |
| The policy picks a signature or SBOM artefact | Sidecar tags entering the catalogue; extend exclusionList | ImageRepository spec |
| Policy has the right image; Git never changes; automation says Ready | No marker matched — wrong file, wrong update.path, malformed comment, or a policy name in a different namespace | Grep the repo for $imagepolicy |
ImageUpdateAutomation not Ready — push rejected | Read-only deploy key, branch protection, or an unsigned commit on a branch requiring signatures | flux logs --kind=ImageUpdateAutomation |
| Git updates but the cluster does not | Not an image-automation problem — the source or Kustomization chain is stalled or suspended | flux get all -A |
| A human’s manual tag edit keeps reverting | The automation is doing its job; the human is editing a managed field | flux suspend image update first |
| Registry returns 429 / “too many requests” | Scan interval too aggressive, or scanning anonymously against a rate-limited public registry | ImageRepository interval and secretRef |
Mutable tags defeat the entire design
If your build pushes :latest, or re-pushes :v1.4 to point at a new artefact, image automation cannot help you and will not tell you so. The failure is silent: the tag string never changes, so the policy’s answer never changes, so the automation never has anything to commit — Git is quietly correct and the cluster is quietly running whatever it happened to pull last. Nothing is broken; the design assumes a tag is a permanent name for one artefact and you have removed that assumption. The fix lives in CI: one unique tag per build, forever. Many registries can enforce it with an immutable-tag setting on the repository, and turning that on is one of the highest-value hours a platform team can spend. Where you need the strongest guarantee, pin the digest too — a digest is content-addressed and cannot be re-pointed at all, which is also what makes signature verification and admission policy meaningful.
A policy that matches nothing
The commonest single failure here, and the one whose symptom is most misread. An ImagePolicy whose filter or range excludes every tag does not quietly fall back to “no change” — it goes not Ready, reporting that it could not determine a latest image, and everything downstream stops. People see “no image found”, assume the registry is unreachable, and debug credentials that are perfectly fine.
Diagnose from the inside out. Confirm the scan is healthy with a sensible tag count, which proves the credentials and repository path. Then take the actual tag you expect to win and test your pattern against it — echo 'main-9f2a1c-1727101800' | grep -E '^main-[a-fA-F0-9]+-[0-9]+$' is a fine sanity check on the exam desktop and immediately exposes a missing anchor or a swallowed backslash. Then check the ordering actually applies to the extracted value: a numerical policy over a substring containing letters will never sort. And with semver, remember that any tag which is not valid semver is invisible to it — a repository full of main-9f2a1c build tags has, from a semver policy’s point of view, no tags at all.
Two different credentials, two different failures
The reflector needs read access to the registry’s tag-listing API; the automation controller needs write access to the Git remote. Separate systems, separate Secrets, separate messages, each unaffected by the other being broken — conflating them wastes real time in an incident. One trap worth naming: a registry the nodes can pull from is not automatically a registry the controller can list from, because kubelet pull credentials and controller scan credentials are different paths. “The pods are running, so the registry works” is not evidence that the scan will.
The robot versus the human
The most confusing failure in a live incident: an engineer edits the image tag in Git to roll something back, and minutes later it is silently changed back. Nothing is broken — the automation is reconciling toward the policy’s answer exactly as configured, and the human is editing a field the robot owns. So suspend before you edit: flux suspend image update <name>, make the change, verify, then decide deliberately whether to resume. Resuming re-applies the policy’s choice, so a genuine rollback means changing the policy too — narrowing the semver range below the bad release, for instance — not just the manifest. And as with every suspend in Flux, the thing that bites is forgetting to resume: a suspended automation looks identical to a working one until somebody notices no deploy has happened for a fortnight. Triage: Delivery has the full decision tree.
Point the automation at the same repository your CI builds from, without a guard, and you have built a perpetual motion machine: the bot pushes a commit, CI sees a push and builds an image, the new image produces a new tag, the policy selects it, the bot pushes again. Three guards, any one sufficient: put a skip marker such as [skip ci] in messageTemplate; configure CI to ignore commits authored by the bot; or — cleanest — keep application source and deployment config in separate repositories so a config commit cannot trigger a build at all.
Rate limits, and the temptation to set every interval to 10s
Every ImageRepository interval is an authenticated call to a registry API, and every one of those APIs meters you. Anonymous pulls from public registries are the most aggressively limited; cloud registries have per-account quotas. Multiply one repository by a ten-second interval by fifty services by three clusters and you have built a small denial-of-service against your own infrastructure — whose likeliest first casualty is not the scan but ordinary pod startup, because throttling hits image pulls too. Sensible defaults: minutes rather than seconds, authenticate even for public images (authenticated limits are usually far higher), keep the exclusionList tight, and reach for flux reconcile image repository when you personally need an answer now.
On a kind cluster with Flux bootstrapped from a personal repo and --components-extra set: 1) Create an ImageRepository for a public image with many tags — ghcr.io/stefanprodan/podinfo works well — and confirm the tag count. 2) Write a semver policy with a wide range, check flux get image policy, then narrow the range until it matches nothing and watch it go not-Ready. Read the exact message. 3) Add a Deployment with a setter marker and an ImageUpdateAutomation pushing to main, and watch the bot commit. 4) Break the marker on purpose — move it to the line above the image: field — and confirm the automation reports success while changing nothing. That silent-success mode is the single most valuable thing here to have seen with your own eyes. 5) Hand-edit the tag in Git and watch it revert; then flux suspend image update and confirm your edit sticks. 6) Switch push.branch to flux-image-updates and confirm nothing deploys until you merge.
Alternatives and when to choose it
☺ Like you’re 10: Everyone agrees the version number should be updated automatically. They disagree about who writes it down and where.
Automated image updates are not unique to Flux — but the approaches differ on the one axis that matters, which is whether the new version reaches Git before it reaches the cluster.
| Approach | How the new tag reaches the cluster | Git stays authoritative? | Best when |
|---|---|---|---|
| Flux image automation | Controllers scan the registry and commit the tag into the repo; the normal reconcile loop deploys it | Always — there is no other write path | You already run Flux and want the loop closed without leaving GitOps |
| Argo CD Image Updater | A separate argoproj-labs component, configured by annotations on the Application; can write back to Git, or write parameter overrides through the Argo CD API | Only in its Git write-back mode | You run Argo CD and accept a less mature add-on — pick the Git mode deliberately |
| Dependency-bot PRs (Renovate and friends) | An external bot opens a pull request bumping the tag | Yes — a PR is a Git change | You want review by default and already run the bot for library and chart versions |
| CI writes the commit itself | The build job clones the config repo and pushes the new tag as its last step | Yes, but CI now holds write credentials to your config repo | Simple setups; the honest fallback when you cannot run extra controllers |
| In-cluster mutation (Keel-style) | A controller patches the live Deployment when it sees a new tag | No — the cluster moves ahead of the repo | Rarely, on a platform that is not doing GitOps at all |
| Mutable tag plus restart | Re-push :latest and restart the pods | No — nothing records which build is live | Never, in anything you are on call for |
The Argo CD comparison, honestly
Argo CD core has no image-updating capability; the function comes from Argo CD Image Updater, a separate argoproj-labs project with its own release cadence and a maturity that has consistently trailed Argo CD itself. It is configured through annotations on the Application rather than dedicated CRDs, so the configuration is less introspectable — no object to kubectl describe, no status telling you which tag it currently believes is newest. It supports comparable selection strategies (semantic version, newest build, alphabetical, digest), so on raw capability the gap is narrower than it is often made out to be.
The real difference is the write-back. Image Updater offers two modes: write the change into Git, or push it through the Argo CD API as a parameter override on the Application. That second mode is convenient and it is a genuine break in the chain — the running state now includes a value no commit contains, and git revert stops being a rollback. Flux does not offer that mode at all, and the absence is deliberate. So “Flux has built-in image automation and Argo CD needs a separate project for it” is fair and exam-correct; “Argo CD cannot do it” is not.
What image automation is not an alternative to
Exam questions probe this boundary, so be precise. It is not a CI system: it never builds, tests, tags or signs an image, so Tekton or a hosted CI still sits upstream and its tagging discipline is a prerequisite. It is not progressive delivery: it changes a tag in Git and stops caring, so canaries and automatic rollback on bad metrics need Flagger or Argo Rollouts on top. It is not supply-chain security: it will happily select a tag nobody signed, which is why verification belongs in admission control with Kyverno and Cosign, where it cannot be bypassed. It is not a promotion pipeline: promoting a tested artefact from staging to prod is expressed by your branch strategy and policy ranges, not by the controller. And it is not the deployer — the Kustomization or HelmRelease still applies the change, so a stalled automation and a stalled reconciler are different problems with different fixes.
Foxy: Let me get this straight. You gave a robot write access to the repository that controls production. That is the plan?
Recon: BEEP. I have write access to one path in one repo, and I can only change strings a human marked with a comment. Everything I do is a signed commit with my name on it.
Timmy: Which is strictly better than what we had, Foxy. Before, the person doing the bump had write access to everything, at 4pm on a Friday, from a laptop.
Benny: And in prod he pushes to a side branch, so it is still a pull request. Dev gets the fast path, prod gets the review. Same controller, one field different.
Gizmo: I simplified it! Everything is :latest now, and I set imagePullPolicy: Always. Zero configuration! 🤑
Timmy: Gizmo. Two of your replicas are running different builds right now and neither is in any commit. When this pages someone at 3am, what exactly do they roll back to?
Dot: Honestly the bit I like is that the deploy shows up as a commit I can read. I know what shipped and when without asking anyone.
Exam relevance and going further
☺ Like you’re 10: The three object shapes and the magic comment have to be in your head — you cannot look them up during the test.
Image automation sits in Domain 2 — GitOps & Continuous Delivery, worth 25% of the CNPE and tied for the heaviest weighting. It is a good discriminator topic: it tests whether you understand why the write-back goes to Git, not merely which YAML field to fill in.
The only permitted documentation during the CNPE is kubernetes.io/docs, kubernetes.io/blog, any docs explicitly linked in a task’s Quick Reference box, and local man pages and /usr/share docs on the exam desktop. Flux is not Kubernetes, so its documentation is off-limits unless a task links it. That means the ImagePolicy shape and — especially — the exact spelling of # {"$imagepolicy": "ns:name"} must come out of your memory. Rehearse the on-cluster fallbacks: kubectl api-resources --api-group=image.toolkit.fluxcd.io, kubectl explain imagepolicy.spec --recursive, and flux create image policy --help. Drill the shapes on Know Cold.
⚖ CNPA vs CNPE — the allowlist is a CNPE mechanic; CNPA has no allowlist because it is fully closed-book — no external resources of any kind. Image automation is unlikely to appear as a CNPA task, but the concept can: that a deployment change must be a change to the declared state, and that a tool mutating the cluster directly has stepped outside GitOps, is exactly the sort of principle CNPA tests from recall.
What to be able to do cold
Without notes: install the two extra controllers and explain why a default bootstrap omits them; write an ImageRepository and know that spec.image carries no tag; write all three ImagePolicy shapes and say which tagging scheme each suits; use filterTags with a named capture and an extract, and explain that the extract is only the sort key while the whole tag is what gets deployed; place a setter marker in all three forms; write an ImageUpdateAutomation with a commit author, a message template and a deliberate push branch; explain why the write-back goes to Git; and diagnose the five headline failures — controllers absent, scan unauthenticated, policy matching nothing, marker not matching, and push rejected for want of a write-capable key.
Rehearse the loop in Practice: GitOps, drill it against the clock in Practice Tasks, keep Command Reference as your cheat sheet, and use the Glossary for vocabulary. Then take the siblings: Flux — Helm for HelmRelease in depth, Flux — Multi-Tenancy for the RBAC and cross-namespace rules that decide who may point an automation at what, Flagger for what happens to the workload after the tag lands, Argo CD for the other half of the GitOps duopoly, and the GitOps lesson for the theory underneath. Reference Architecture shows where the whole chain sits.
Official sources — for study time, not exam time
Read these before the exam: the image-automation guide at fluxcd.io/flux/guides/image-update, the component API references at fluxcd.io/flux/components/image, the source repositories at github.com/fluxcd/image-reflector-controller and github.com/fluxcd/image-automation-controller, and the vendor-neutral principles at opengitops.dev. For the range syntax the semver policy uses, semver.org is worth reading once properly.
1. Name the two controllers and the three CRDs, and say which talks to the registry and which to Git. 2. Why does image automation commit to Git instead of patching the live Deployment — what specifically breaks if it did the latter? 3. Your tags look like main-9f2a1c-1727101800. Which ordering strategy, and what else must you configure for it to work at all? 4. With filterTags, is the extracted value or the whole tag what ends up in your manifest? 5. Write the setter marker that rewrites only the tag portion of a Helm values field, for a policy named checkout in flux-system. 6. Every Flux object is healthy, but the automation is not Ready and the log mentions permissions. Most likely cause on a freshly bootstrapped cluster? 7. A colleague hand-edits an image tag in Git and it keeps reverting. What do you run first? 8. Your policy is not Ready with “no image found”, yet the scan reports 400 tags. Where is the fault?
Check your answers
image-reflector-controllerownsImageRepositoryandImagePolicyand reads the registry;image-automation-controllerownsImageUpdateAutomationand reads and writes Git. Neither is installed by a default bootstrap.- A controller patching the live Deployment puts the cluster ahead of the repository: the running state is no longer described by any commit, so
git revertis not a rollback,git logis not a deployment history, and the drift-correction loop would fight the patch or the patch would defeat the loop. Committing to Git keeps the reconciler the only writer to the cluster. numericalwithorder: asc. On its own it cannot work, because the whole tag is not a number — addfilterTagswith a pattern capturing the trailing timestamp and anextractselecting it as the sort key.- The whole tag. The extract is only the sort key used to decide which tag wins; the winner is written in full.
# {"$imagepolicy": "flux-system:checkout:tag"}— on the same line as thetag:value, as a genuine YAML comment with straight double quotes.- The deploy key created by
flux bootstrapis read-only by default and the automation controller needs to push. Re-bootstrap with--read-write-key, or give the key write access in the Git host. Branch protection rejecting the push is the other candidate. flux suspend image update <name>— the automation is correctly reconciling toward the policy’s answer and will keep reverting a manual edit. A real rollback also means changing the policy, not just the manifest.- In the policy, not the registry. The scan is healthy, so either
filterTags.patternmatches none of those 400 tags, the semver range excludes them all, or the tags are not valid semver and you used asemverpolicy.