GitOps Workflows
GitOps is the idea that flips a whole platform from “humans run commands at the cluster” to “the cluster continuously pulls its own truth from Git.” You describe what you want in a Git repository; a controller running inside the cluster watches that repo and never rests until reality matches. This lesson is the heart of one of the exam’s two joint-largest domains at 25% — the four principles, the reconciliation loop and drift, Argo CD vs Flux, how to lay out your repos, and how to scale to many apps and clusters with ApplicationSets.
Imagine you write down exactly how your bedroom should look on a big poster: bed made, toys in the blue box, books on the shelf. Now imagine a tireless robot who checks the poster against your real room every minute and quietly fixes anything that’s wrong — if your little brother moves a toy, the robot puts it back. GitOps is that poster (a Git repo) plus that robot (a controller). You never tidy the room by hand again; you just change the poster, and the robot makes the room match.
What GitOps actually is
☺ Like you’re 10: Instead of walking to the cluster and telling it what to do, you write it down once, and a helper inside the cluster keeps things matching what you wrote.
For years, teams deployed by pushing commands at a cluster: someone (or a CI job) ran kubectl apply or helm upgrade from a laptop or a pipeline. It works, but it has three quiet problems: the cluster’s real state can silently drift from what anyone intended, you need to hand powerful credentials to your CI system, and there’s no single place that answers “what is supposed to be running right now?” GitOps fixes all three by making Git the single source of truth and putting a reconciler inside the cluster that pulls from Git and applies it.
From “push” to “pull”
The mental shift is from push to pull. In a push model, an external actor reaches into the cluster and changes it. In a pull model, an agent inside the cluster reaches out to Git, notices the difference, and updates itself. That inversion is why GitOps is more secure (the cluster’s credentials never leave it — your CI system never needs cluster access) and more reliable (there’s always an authoritative answer to “what should be here?”).
GitOps = declarative desired state in Git + a controller that continuously reconciles the cluster to match it. Deployments stop being an action you take and become a state you declare. To ship, you change Git; to roll back, you revert a commit.
Desired state vs actual state
Two words run through this entire lesson. Desired state is what you wrote in Git — the manifests, the versions, the replica counts. Actual state is what’s really running in the cluster right now. Recon the Robot’s only job is to shrink the gap between them to zero, over and over, forever. When they match, the app is Synced. When they don’t — because someone hand-edited a resource, or a node died — the app is OutOfSync, and Recon acts.
“I don’t open a deploy tool or ask anyone for cluster access. I open a pull request that bumps my image tag from 1.4.2 to 1.4.3. A teammate approves, it merges, and ninety seconds later it’s in prod — and if it misbehaves, we just revert the PR. My entire deployment interface is the thing I already use all day: Git.”
The four OpenGitOps principles
☺ Like you’re 10: There are four rules that make something “really” GitOps and not just “we keep some YAML in Git.”
The CNCF’s OpenGitOps project boils GitOps down to four principles. The exam expects you to recognise them and — more importantly — to configure a system that honours them. A system is only truly GitOps if all four hold:
| # | Principle | What it means in practice |
|---|---|---|
| 1 | Declarative | The whole desired state is expressed declaratively — you describe the end state (Kubernetes manifests, Helm values, Kustomize), not the steps to get there. |
| 2 | Versioned & Immutable | Desired state is stored so it’s versioned, immutable, and keeps a complete history — that’s Git. Every change is a commit you can audit or revert. |
| 3 | Pulled Automatically | Software agents automatically pull the desired state from the source — no human runs apply. |
| 4 | Continuously Reconciled | Agents continuously observe actual state and reconcile it toward desired state — forever, not just once at deploy time. |
Notice how the last two rule out common half-measures. “We run kubectl apply from a GitHub Actions job on every merge” satisfies 1 and 2 but fails 3 and 4: it’s pushed, and it only reconciles at merge time, so drift between merges goes uncorrected. That’s CI-driven deployment, not GitOps.
Reconciliation & drift — Recon’s loop
☺ Like you’re 10: The robot runs the same tiny loop forever: look at the poster, look at the room, fix the difference. That’s it.
The reconciliation loop is the beating heart of GitOps (and, as you’ll see in Platform APIs & Operators, of every Kubernetes controller). It’s level-triggered, not edge-triggered: it doesn’t react to a one-off event and hope it worked — it repeatedly compares the desired level to the actual level and drives toward the target, so it’s naturally self-healing.
How often the loop runs is configurable, and the two engines expose it differently. Argo CD refreshes each Application against Git on a periodic interval — three minutes by default — and a repository webhook makes that near-immediate. Flux makes the cadence explicit on every object: each GitRepository, Kustomization and HelmRelease carries its own interval. Drift inside the cluster is usually caught sooner than either interval suggests, because both controllers watch live resources rather than waiting for the next poll.
Self-heal and prune
Two switches turn the loop from “informational” to “enforcing.” Self-heal means when actual drifts from desired (someone runs kubectl scale by hand), the reconciler reverts it back to Git. Prune means when you delete a manifest from Git, the reconciler deletes the corresponding live resource — so Git deletions are real deletions, not orphans. Here’s an Argo CD Application with both enabled:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: checkout
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/acme/platform-config.git
targetRevision: main
path: apps/checkout/overlays/prod # a Kustomize overlay
destination:
server: https://kubernetes.default.svc
namespace: checkout
syncPolicy:
automated:
prune: true # delete live resources removed from Git
selfHeal: true # revert out-of-band (drift) changes back to Git
syncOptions:
- CreateNamespace=trueSelf-heal makes kubectl edit on a managed resource useless — your change is reverted within seconds, which surprises engineers debugging in prod. That’s the point (no snowflakes), but document it loudly, and teach people to “fix it in Git.” Likewise, turning on prune without understanding what’s in Git can delete live resources fast — review the diff before enabling it on a busy cluster.
Argo CD vs Flux — the two engines
☺ Like you’re 10: There are two popular “tireless robots” to choose from. They do the same job in slightly different styles — one comes with a nice dashboard, the other is a set of small Lego pieces.
Two CNCF-graduated projects dominate GitOps, and the exam tool list names Argo and Flux. You should be able to read and configure both. They share the philosophy but differ in shape.
Argo CD — application-centric, with a UI
Argo CD models everything as an Application (or ApplicationSet) custom resource and ships a polished web UI and CLI that visualise sync status, diffs, and the resource tree. Internally it runs three main pieces: an API/UI server, a repo-server (renders manifests from Git — Kustomize, Helm, plain YAML), and the application-controller (the reconciler that compares and syncs). Handy patterns include App-of-Apps (one Argo app that points at a folder of other Argo apps) and sync waves (ordering resources during a sync).
Flux — a toolkit of controllers
Flux is the “GitOps Toolkit”: a set of small, composable controllers you assemble. source-controller fetches Git/Helm/OCI sources; kustomize-controller and helm-controller reconcile them; notification-controller handles alerts and webhooks; and the image-reflector/image-automation controllers can bump image tags in Git automatically. It’s API-driven and UI-optional — a natural fit if you want to compose your own platform primitives. The same checkout app in Flux:
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: platform-config
namespace: flux-system
spec:
interval: 1m
url: https://github.com/acme/platform-config.git
ref:
branch: main
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: checkout
namespace: flux-system
spec:
interval: 10m
sourceRef: { kind: GitRepository, name: platform-config }
path: ./apps/checkout/overlays/prod
prune: true
wait: true # wait for resources to become Ready| Dimension | Argo CD | Flux |
|---|---|---|
| Core abstraction | Application / ApplicationSet | GitRepository + Kustomization / HelmRelease |
| UI | Rich built-in web UI & CLI | No official UI (CLI + optional dashboards) |
| Shape | One integrated app (application-centric) | Composable toolkit of controllers |
| Multi-tenancy | Projects, RBAC, SSO in the UI | Namespaced CRs + Kubernetes RBAC |
| Image updates | Argo CD Image Updater (separate) | Built-in image automation controllers |
| Feels like | A deployment product | Platform-building primitives |
Don’t agonise over “which is better.” For the exam, know that both implement the four principles; be able to point an app at a repo path, enable prune/self-heal, and read sync status. For a real platform, pick the one whose shape matches your team — Argo’s UI for broad developer visibility, Flux’s toolkit for composing bespoke platform APIs.
Structuring your Git repositories
☺ Like you’re 10: Keep the “how to build the app” stuff and the “what’s running where” stuff in separate drawers, and keep one folder per environment so dev and prod can’t clobber each other.
Repo layout is where GitOps projects live or die. Two rules carry you a long way. First, separate the app/source repo (code, Dockerfile, unit tests, CI) from the config repo (the manifests the reconciler watches). Second, separate environments with Kustomize overlays (a shared base plus per-env patches) or Helm values, so promoting dev → staging → prod is a small, reviewable diff.
platform-config/ # CONFIG repo — watched by Argo CD / Flux ├── apps/ │ └── checkout/ │ ├── base/ # shared manifests │ │ ├── deployment.yaml │ │ ├── service.yaml │ │ └── kustomization.yaml │ └── overlays/ │ ├── dev/ # env-specific patches (replicas, image tag) │ ├── staging/ │ └── prod/ ├── infrastructure/ # cluster add-ons: ingress, cert-manager, mesh └── tenants/ # one folder per team → fanned out by an ApplicationSet checkout/ # separate APP repo — code + Dockerfile + CI pipeline
A Git repo is world-readable to everyone who can clone it and lives forever in history. Never commit raw secrets. Use Sealed Secrets (encrypt to a cluster-specific key so only the controller can decrypt), the External Secrets Operator (sync from Vault / cloud secret managers into the cluster), or SOPS (encrypt values in-repo; Flux decrypts on apply). The reference to a secret lives in Git; the plaintext never does.
Scaling with ApplicationSets
☺ Like you’re 10: Instead of writing one poster per room by hand, you write one template and a machine stamps out a poster for every room automatically.
Hand-writing an Application per team per cluster doesn’t scale. Argo CD’s ApplicationSet templates them from generators — a list generator (explicit items), a Git directory/file generator (one app per folder in a repo), a cluster generator (one app per registered cluster), a matrix (combine two generators — e.g. every app × every cluster), and a pull request generator (an ephemeral preview environment per open PR). This is how a platform onboards a new team by adding one folder:
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: tenants
namespace: argocd
spec:
goTemplate: true # Go text/template syntax (the current form)
goTemplateOptions: ["missingkey=error"]
generators:
# one Application per folder under tenants/
- git:
repoURL: https://github.com/acme/platform-config.git
revision: main
directories:
- path: tenants/*
template:
metadata:
name: '{{.path.basename}}' # e.g. "payments", "search"
spec:
project: default
source:
repoURL: https://github.com/acme/platform-config.git
targetRevision: main
path: '{{.path.path}}' # full path; .path.basename is the last segment
destination:
server: https://kubernetes.default.svc
namespace: '{{.path.basename}}'
syncPolicy:
automated: { prune: true, selfHeal: true }
syncOptions: [ CreateNamespace=true ]Now “onboard the payments team” is a one-line change — create tenants/payments/ — and the platform provisions their namespace and app automatically. That is the exact moment GitOps stops being a deploy tool and becomes a self-service platform capability.
GitOps for the whole platform, not just apps
☺ Like you’re 10: The same trick works for the plumbing too — databases, networks, the cluster’s own add-ons — not only your apps.
The exam competency says “Application and Infrastructure deployment,” and that word matters. Because everything in Kubernetes is a declarative API object — including Custom Resources from Crossplane that represent cloud databases and networks — you can put your infrastructure under the same reconciler. Cluster add-ons (ingress controller, cert-manager, the service mesh), tenant setup, and even cloud resources all become folders in the config repo. One control plane, one audit log, one way to roll back. That unification — apps and infrastructure reconciled from Git — is the backbone of the platform you’ll assemble across the rest of this course.
You’ll build the left half of that picture — the pipeline and the safe rollout — next, in CI/CD & Progressive Delivery.
On a throwaway cluster (kind or minikube), install Argo CD, then point an Application at a public Git repo path with selfHeal: true. Watch it sync. Now kubectl scale the deployment to a different replica count by hand and watch Argo revert you within seconds — that’s reconciliation defeating drift. Finally, delete a manifest from your fork and confirm prune removes the live resource. Three tiny experiments, and the whole domain clicks.
Foxy: So GitOps is just… running kubectl apply from a pipeline on every merge, right?
Benny: Close, but no. That’s push, and it only reconciles at merge time. Real GitOps has an agent inside the cluster that pulls and reconciles continuously.
Recon: BEEP. I never stop. Desired state, actual state, diff, apply. Someone hand-edits a Deployment? I revert it in twelve seconds.
Gizmo: Fine, but hardcode the database password right in the manifest — it’s so much faster. Who’s gonna look? 🤑
Timmy: The entire internet, Gizmo — Git history is forever. Sealed Secrets or External Secrets. The reference goes in Git; the plaintext never does.
Dot: Honestly I don’t care how the robot works — I just love that “deploy” is now “open a PR,” and “roll back” is “revert it.”
GitOps gives you the reconciled foundation. The other half of the 25% domain is the delivery machinery that feeds it — pipelines and safe rollouts — which is exactly where Benny and Pip take you next.
1. In one sentence, how does GitOps differ from running kubectl apply in CI? 2. Name the four OpenGitOps principles. 3. What do prune and selfHeal each do, and what surprising behaviour does self-heal cause? 4. Give one correct way to handle a secret in a GitOps repo. 5. Which ApplicationSet generator would you use to give every open pull request its own preview environment?
Check your answers
- GitOps puts an agent inside the cluster that continuously pulls desired state from Git and reconciles it; CI-driven
applypushes once at merge time and doesn’t correct later drift. - Declarative; Versioned & Immutable; Pulled Automatically; Continuously Reconciled.
prunedeletes live resources that were removed from Git;selfHealreverts out-of-band (drift) changes back to Git — which makes a manualkubectl editget reverted within seconds.- Any of: Sealed Secrets (encrypt to a cluster key), External Secrets Operator (sync from Vault / a cloud secret manager), or SOPS (encrypt values in-repo, decrypt on apply). Never commit plaintext.
- The pull request generator.