GitOps Philosophy
The CGOA blueprint teaches four principles you can recite in under a minute: declarative, versioned and immutable, pulled automatically, continuously reconciled. That's enough to pass an associate exam. It is not enough to explain why the model holds together, why a pull-based reconciler is safer than a push-based pipeline for reasons that have nothing to do with taste, or why a system built around GitOps treats drift — the very thing that sounds like the failure mode — as closer to its fuel supply than its enemy. This page assumes you already know the four principles cold; if you don't, start with CGOA — the exam and come back. What follows is the model underneath the model: GitOps read as a closed-loop control system borrowed wholesale from control theory, the level-triggered design that makes it resilient to lost events, the trust-boundary argument for pull over push in full, and the places the philosophy genuinely strains under a real workload.
Think about a thermostat, not a light switch. A light switch is push: you flip it once, and the room stays lit only because nothing else touches it — if a gust of wind blows the bulb loose, the room stays dark until a person notices and flips the switch again. A thermostat is different. You set one number — the temperature you want — and the thermostat never stops checking: it reads the actual temperature over and over, compares it to your number, and turns the furnace on or off, forever, without caring whether the room got cold because a window opened, a vent got blocked, or nobody ever told it anything changed. GitOps builds every deployment like the thermostat, not the light switch. The "wanted" number lives in Git, and something is always checking.
What the badge doesn't ask: the model behind the four principles
☺ Like you're 10: The exam wants you to recite four rules. This page wants you to see why they're one machine, not four separate facts you happened to memorize together.
Quick recap, at exam depth, so this page stands on solid ground: Declarative — the state store describes the end state, not a sequence of steps to get there. Versioned and Immutable — every change is a new, addressable version with a complete history, never an in-place overwrite. Pulled Automatically — software agents inside the managed system pull changes; nothing external pushes into it. Continuously Reconciled — agents observe actual state and converge it toward desired state, indefinitely, not once. The Platform Engineering course's own CGOA overview and GitOps workflows lesson cover these four at exactly the depth the exam rewards, with the classic failure-mode distractors worked out in full — this page doesn't repeat that ground, it stands on it.
What none of that vocabulary explains is why this particular combination of four rules produces a system that self-heals, why "pulled" and "continuously reconciled" are separate principles rather than one restated twice, or what actually goes wrong when a team implements three of the four and calls it GitOps anyway. Those are the questions this page answers, and the answers come from a field a lot older than Kubernetes: control theory.
The reconcile loop as a closed-loop control system
☺ Like you're 10: Every reconciler is a thermostat with three working parts: something that measures, something that compares, something that acts.
Control theory has a standard vocabulary for exactly the thermostat above, and it maps onto a GitOps reconciler almost embarrassingly well. A setpoint is the target — in a thermostat, the temperature dial; in GitOps, the desired state committed to the store. A sensor measures the actual condition of the plant (control theory's word for the system being controlled) — in Kubernetes terms, the API server's watch stream and informer caches, continuously reporting what's actually running. A comparator subtracts observed from desired to produce an error signal — this is drift, formally: e(t) = desired(t) − observed(t). A controller turns that error signal into a corrective action, and an actuator applies it back to the plant — in GitOps, both roles collapse into the reconciler itself, which computes the diff and issues the API calls that shrink it. Wire the sensor's output back into the comparator's input and you have a closed loop: a system whose next action always depends on its own most recent effect. Wire nothing back — run a script once and walk away — and you have an open loop, sometimes called feedforward control: it acts on its best guess and never finds out whether the guess was right.
This is also the honest answer to why Pulled Automatically and Continuously Reconciled are two separate principles rather than one restated twice. A system could pull automatically once a day and never check again afterward — that's still automatic pulling, and it would still satisfy the letter of principle three. It fails principle four, because a control loop that samples once and stops is exactly the open-loop script in the dashed box above, just with a longer delay before it runs. Continuous reconciliation is the specific, separate claim that the loop stays closed forever, not that it merely starts by pulling instead of being pushed to.
Level-triggered, not edge-triggered: why the loop doesn't need to remember what it missed
☺ Like you're 10: A doorbell only rings if someone presses it. A guard doing rounds checks the door whether or not anyone rang anything.
CGOA's patterns domain draws an axis between pull and event-driven triggers — what wakes the reconciler up. There's a related but genuinely deeper design property underneath it, borrowed from interrupt handling and general control-systems design rather than from GitOps specifically: whether a system is edge-triggered or level-triggered. An edge-triggered system reacts to a transition — a notification, a webhook, "this commit just happened." Its correctness depends on actually receiving that notification; drop the webhook, and the system has no other mechanism telling it anything is wrong. A level-triggered system instead samples the current state on a schedule, independent of any specific event — it doesn't ask "did something happen since I last checked," it asks "does reality match intent, right now," and it asks that question again next cycle regardless of what it found or missed last time.
This is why the two axes are compatible answers to two different questions, not competing choices. Every serious reconciler is event-driven for latency — a watch event on a changed resource wakes it immediately, so convergence doesn't wait for the next scheduled tick — and level-triggered for correctness — it also resyncs on a fixed period regardless of events, so a lost webhook, a missed watch event during a network partition, or a change nobody notified it about all get caught anyway, on the very next pass. The event is an optimization. The periodic resync is the guarantee.
Real controllers make this concrete in code. Below is the shape of a Kubernetes-style reconcile function — deliberately kept generic, not tied to any one operator framework — that treats "an event just fired" and "my resync period just elapsed" as the exact same call, because from inside the function they are indistinguishable:
// Reconcile is invoked on every watch event for the resource AND on a
// fixed resync period — the function body doesn't know or care which one
// triggered this particular call, and that's the whole point.
func (r *Reconciler) Reconcile(ctx context.Context, req Request) (Result, error) {
desired, err := r.loadDesiredState(req.Name) // read from the state store
if err != nil {
return Result{}, err
}
observed, err := r.getLive(ctx, req.Name) // read from the live API
if err != nil {
return Result{}, err
}
if drift := compare(desired, observed); drift.NonEmpty() {
if err := r.apply(ctx, desired); err != nil {
return Result{RequeueAfter: backoff}, err // retry sooner on failure
}
}
// Ask to be called again even if nothing else happens in between —
// this line is what makes the loop level-triggered, not edge-triggered.
return Result{RequeueAfter: 3 * time.Minute}, nil
}On a throwaway cluster, block outbound access from your GitOps agent's namespace for a few minutes — simulating a lost webhook or a network partition. While it's blocked, edit a managed resource by hand so real drift exists. Restore access and watch what happens on the next scheduled reconcile pass, not the moment you restored the network. Nothing needed to "catch up" on missed events, because nothing was ever remembering them in the first place. The stuck-sync drill walks a closely related failure mode step by step.
Why pull beats push: the trust-boundary argument in full
☺ Like you're 10: Push means a stranger's key can open your door from outside. Pull means your own door decides when to check the street.
CGOA teaches "pulled automatically" as a rule: agents pull, nothing pushes in. The exam-scoped reason is usually left implicit. Spelled out: in a push model, one central CD system holds the write credentials for every environment it deploys to, and it initiates the connection into each one. In a pull model, each environment holds only a credential to read a shared state store, and it is the one that initiates outward — nothing external ever holds a key that opens that environment's door.
That single reversal changes the blast radius of a compromise. Pop the central push system in the first model and you have write access to every environment it was trusted to reach, in one step — that system is, by construction, the single highest-value target in the whole architecture. Pop a reconciler running inside one cluster in the second model and you've gained whatever that one cluster's own trust boundary already grants — there is no push path from there into any other environment, because no such path exists at all, compromised or not. The second, quieter argument concerns visibility of failure. A push pipeline that fails partway through often fails silently from the target's point of view: a deploy step errors out mid-run, and unless a person is watching that specific pipeline's dashboard, the cluster simply sits in whatever half-applied state the failed run left it in, with no signal from the cluster itself that anything is wrong. A reconciler that can't reach its state store, or can't converge, is a condition the target can report at the point of truth — a stale sync status, an unreachable-source health check — rather than something inferred from a CI system that has already moved on to the next job.
| Property | Push (CD system → cluster) | Pull (cluster ← state store) |
|---|---|---|
| Where write credentials live | Centrally, in the CD system, for every target it reaches | Locally, inside each target's own trust boundary |
| Blast radius of one compromise | Every environment the CD system can reach | The one environment whose reconciler was compromised |
| Failure when the link breaks mid-run | Often silent — a half-applied state, no self-report | Observable at the target — a stale or unreachable sync status |
| Who initiates the connection | An external system, inbound into the cluster's network | The cluster's own reconciler, outbound |
Be honest about the limit of this argument, though: it's a property of the trust boundary, not a magic property of the word "pull." An external reconciler managing many clusters from one place — a common, legitimate pattern for fleet-wide GitOps — reintroduces a version of the exact centralization risk pull was meant to avoid, because that one external system again holds credentials reaching into many environments. Pull shrinks blast radius specifically when the reconciler runs inside the boundary it manages; how far you centralize reconciler placement is its own trade-off, covered at the tooling-pattern level in the CGOA overview's own treatment of in-cluster versus external reconcilers.
Drift as a first-class concept, not a failure to eliminate
☺ Like you're 10: The moment you commit a change, the live system is already "wrong" compared to Git — and that's supposed to happen.
Most engineers meet the word "drift" as a warning label — something a dashboard flags in red, something to be eliminated. Read as the error term in a control loop, though, e(t) = desired − observed, drift isn't an anomaly the system tolerates; it's the exact quantity the whole loop exists to act on. No real control system holds its error term at a permanent, literal zero — a thermostat is very slightly above or below its setpoint essentially all the time, correcting continuously rather than sitting motionless at a perfect number. The instant you commit a new desired state to the store, the live system is, by definition, in drift relative to it, right up until the next reconcile pass closes the gap. Zero drift, forever, isn't the healthy steady state to aim for — bounded, promptly-converging drift is.
A GitOps-managed system reporting zero drift, permanently, for weeks, isn't necessarily healthy — it might mean nobody is changing anything, or it might mean the reconciler has quietly stopped running and there's nothing left to notice the difference. Drift that appears and reliably converges is the sign of a loop that's actually alive; the number to watch is how fast it closes, not whether it's ever exactly zero.
Once drift is a first-class signal rather than a binary bad/good flag, the next skill is classifying which kind you're looking at, because conflating them is where real setups break. Convergence-lag drift is transient and expected — the gap between committing a change and the next reconcile pass closing it; it needs nothing but patience. Imperative drift is a human bypassing the store — a direct kubectl edit or kubectl scale — and the loop is supposed to treat it exactly like any other error term and revert it; this is what "self-heal" means in practice. External-owner drift is the subtle one: a different, entirely legitimate controller — a HorizontalPodAutoscaler changing replicas, a mutating webhook filling in a default, a cloud controller populating a status field — is intentionally changing a field your manifest also declares. Treat that the same as imperative drift and you get a fight: the HPA sets replicas, the reconciler reverts them to the manifest's number, the HPA sets them again, forever, both sides burning API calls to lose the same argument repeatedly. Tooling needs an explicit way to say "a different owner has this field" — in Argo CD, that's ignoreDifferences:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: checkout
spec:
# ...source, destination, syncPolicy as usual...
ignoreDifferences:
- group: apps
kind: Deployment
jsonPointers:
- /spec/replicas # owned by the HPA, not by this manifest — tells
# the reconciler "this drift is not drift"Reading a real manifest and correctly sorting a given field into "will converge on its own," "should be reverted," or "is owned elsewhere and must be excluded" is a genuinely underrated skill that no multiple-choice question tests directly, and it's the difference between a GitOps setup that fights its own autoscaler indefinitely and one that doesn't.
Where the philosophy strains
☺ Like you're 10: No control system is free — a loop that reacts to everything can also fight things it shouldn't, and a rule that works for repeatable actions doesn't cover a step that's only supposed to happen once.
Periodic sampling has a consistency cost the model doesn't hide, only defers: between reconcile passes, the live system can observably diverge from desired state, and for most workloads — a stale replica count for ninety seconds — that's a fine trade against the simplicity of the loop. For some invariants it isn't. A security-relevant misconfiguration that exists for even the gap between two reconcile passes has already been a misconfiguration, whether or not the next pass later corrects it. That's the honest reason a mature GitOps stack pairs a reconciler with real-time admission control rather than leaning on reconciliation alone for anything safety-critical: admission control blocks a bad object from ever being written in the first place, at the moment of the API request, instead of noticing it after the fact on the next cycle. Recon reconciles toward correctness after the fact; Timmy's policy-as-code layer and Kyverno prevent incorrectness from landing at all — two different controls, not competing implementations of the same one.
The model also quietly assumes every actuator action is idempotent — safe to repeat with no side effect beyond convergence. Most Kubernetes API operations genuinely are: applying the same Deployment spec twice does nothing the second time. A one-off database migration Job, or any script with a real side effect, is not naturally idempotent, and doesn't fit the "reapply forever" model cleanly — running it on every reconcile pass would be actively harmful, not merely redundant. GitOps tooling has to reach for an explicitly imperative escape hatch to handle this honestly — Argo CD's sync hooks and Helm's hook phases (PreSync, Sync, PostSync) exist specifically to say "run this once, on this transition, not on every pass," which is, worth naming plainly, imperative machinery deliberately bolted onto a declarative model rather than a gap the model pretends doesn't exist.
Turn on aggressive self-heal before you've told the reconciler which fields belong to another owner, and you'll get the HPA fight described above the moment autoscaling kicks in. The same risk shows up mid-rollout: a raw Deployment's replica count changing gradually during a canary looks, to a naive reconciler, identical to drift that needs reverting. This is exactly why progressive delivery is expressed as its own resource — the Argo ecosystem and Argo Rollouts — rather than as raw replica edits the top-level reconciler would otherwise "helpfully" stomp back to the manifest's static number.
"We had a bad deploy at 2am and I did the thing you're not supposed to do — hand-edited the image tag straight on the live Deployment to roll back fast, no PR, no time. It worked for about ninety seconds. Then Argo CD's next sync pass saw my hand-edit as drift against the still-broken tag in Git and cheerfully reverted my fix back to the bad version, because as far as the reconciler was concerned, I was the incident, not the person fixing one. The rollback had to go through Git after all — reverting the commit, not the running Pod — which is obvious in hindsight and was not obvious at 2am."
Foxy: Recon — level-triggered, edge-triggered, pull, push — I got lost somewhere around "closed-loop." Plain terms?
Recon the Robot: BEEP. A push script runs once and hopes. I run forever and check. Sensor, comparator, actuator, on a loop — not a script with an ending.
Gizmo the Gremlin: Or — hot take — just wire a webhook straight from the merge to a deploy script. Instant! No loop required! 😈
Recon the Robot: That's edge-triggered with no fallback, Gizmo. Drop one webhook and the system stays wrong forever, with nothing telling anyone. I would rather sample and actually know.
Timmy the Turtle: And even Recon's loop only samples every few minutes. Fine for a replica count. Not fine for a policy violation I need blocked the instant it's attempted, not corrected after the fact.
Recon the Robot: Agreed. My job is convergence. Your job is prevention. We are not the same control, Timmy, and a system running only mine is missing half its immune system.
Foxy: So — level-triggering for resilience, admission control for anything that genuinely can't wait a cycle. Got it.
1. In control-theory terms, name the four parts of the GitOps reconcile loop and what plays each part — setpoint, sensor, comparator/error signal, actuator. 2. What's the difference between edge-triggered and level-triggered reconciliation, and why does a real controller usually want to be event-driven and level-triggered rather than picking just one? 3. Give one concrete reason a pull model shrinks blast radius compared to a push model, beyond "it's the rule the exam wants." 4. Name the three kinds of drift discussed on this page, and explain why treating HPA-owned replica drift the same as human kubectl edit drift causes a problem. 5. Why can't periodic reconciliation alone guarantee a security-relevant bad state never exists, even briefly — and what pairs with it to close that gap? 6. Give one example of an imperative escape hatch a declarative GitOps system still needs, and explain why the model can't avoid it.
Check your answers
- Setpoint: desired state in Git / the state store. Sensor: the live API's watch/informer stream reporting observed state. Comparator: the diff between desired and observed, producing an error signal (drift). Actuator: the reconciler's apply step, which the controller role also occupies in GitOps — reconciler and actuator collapse into one component.
- Edge-triggered reacts only to a notification of change and has no other way of noticing if that notification is lost; level-triggered samples current-vs-desired state on a fixed cadence regardless of any specific event, so it self-corrects even after a missed notification. A real controller wants both: event-driven for low latency (react immediately when told), level-triggered for correctness (never depend on the notification actually arriving).
- In a pull model, write credentials for an environment live only inside that environment's own trust boundary and nothing external ever holds a key into it — so compromising the reconciler in one cluster grants only that cluster's own access, with no push path into any other environment, compromised or not.
- Convergence-lag drift (transient, closes on the next pass), imperative drift (a human bypassed the store — should be reverted), and external-owner drift (a different legitimate controller, like an HPA, owns a field the manifest also declares). Treating external-owner drift like imperative drift causes a fight: the external controller sets the field, the reconciler reverts it to the manifest's value, the external controller sets it again — an endless loop, unless the field is explicitly excluded (e.g. Argo CD's
ignoreDifferences). - A bad state can exist for the entire gap between two reconcile passes before the loop notices and corrects it — and for a security-relevant misconfiguration, existing at all, even briefly, is already the problem. Real-time admission control (e.g. Kyverno) pairs with reconciliation by blocking the bad object at write time, instead of catching it after the fact on the next cycle.
- Sync/Helm hooks (
PreSync,Sync,PostSync) for one-off actions like a database migration Job. The model assumes every actuator action is idempotent — safe to repeat — but a genuine one-time side effect isn't, so re-applying it on every reconcile pass would be actively harmful, forcing an explicit imperative escape hatch inside an otherwise declarative system.