Argo CD

Every deployment mechanism this course has covered so far — the deploy stage inside CI/CD pipelines, a plain helm upgrade --install run straight from a Jenkins or GitHub Actions job — is push-based: a pipeline running somewhere outside the cluster authenticates in and pushes a change the moment a build finishes. Argo CD is a GitOps continuous delivery tool for Kubernetes that inverts that relationship. An agent running inside the cluster continuously pulls the desired state from a Git repository and reconciles the live cluster to match it — on its own schedule, with its own credentials, whether or not a pipeline ever ran at all. Git becomes the single source of truth for what should be running, not just a place code happens to live before something else pushes it somewhere.

GitOps: the pattern this page introduces

☺ Like you're 10: Instead of a pipeline shoving a change into the cluster once, an agent living inside the cluster keeps checking Git and copying whatever it finds there, forever.

The term GitOps was coined by Weaveworks in 2017, and the community since organized around a short list of principles worth stating precisely, because Argo CD is one concrete implementation of all four: the desired state of the system is described declaratively; that description is stored in Git, so it is versioned and immutable — every change is a commit, with a full history and an author; approved changes are applied automatically by software, not by a person running a command; and the live system is continuously verified against that desired state, with drift detected and, optionally, corrected without a human noticing first. None of that requires Kubernetes specifically — but Kubernetes' own declarative, reconciling API made it the natural first home for the pattern, which is exactly why Argo CD exists.

Argo CD itself began inside Intuit (via its 2018 acquisition of Applatix), was open-sourced the same year, and — together with its sibling projects Argo Workflows, Argo Events, and Argo Rollouts — became a CNCF incubating project in 2020 and graduated in December 2022, putting it in the same top tier of CNCF maturity as Kubernetes and Prometheus themselves. The full definition is in the glossary; this page assumes it and goes straight to how Argo CD actually implements it.

Architecture: the controllers behind the reconciliation loop

☺ Like you're 10: One piece clones the recipe, one piece cooks it into real ingredients, and one piece keeps tasting the dish and fixing it until it matches.

Argo CD installs as a handful of Deployments in its own argocd namespace, each with a narrow job. argocd-server is the API — a single gRPC/REST service backing both the web UI and the argocd CLI, and the thing that authenticates you, whether via a built-in admin account or SSO through argocd-dex-server (an OIDC bridge to GitHub, Okta, or any standard identity provider). argocd-repo-server does the actual rendering: it clones your Git repository and turns whatever's in it — plain YAML, a Helm chart run through helm template, a Kustomize overlay run through kustomize build, or a config-management plugin for anything else — into a flat list of Kubernetes manifests, then caches the result. And argocd-application-controller is the reconciler itself: it watches every Application custom resource, diffs the repo-server's rendered manifests against the live cluster, and — depending on sync policy — either reports the difference or applies it. redis sits underneath as a shared cache so the controller isn't re-listing every watched resource from the Kubernetes API on every pass, and a separate argocd-applicationset-controller handles the app-of-apps-at-scale pattern covered below.

Two states come out of every reconciliation, and they're easy to conflate but answer different questions. Sync status — Synced, OutOfSync, or Unknown — asks "does live match Git?" Health status — Healthy, Progressing, Degraded, Suspended, Missing, or Unknown — asks "is what's running actually working?" A Deployment can be perfectly Synced — every field matches Git exactly — while its pods crash-loop on a bad readiness probe, which reads as Degraded. Argo CD ships built-in health logic for the standard kinds plus a few Kubernetes-ecosystem extras (Ingress, PVC, Argo Rollouts), and for a custom resource it doesn't understand out of the box, you supply a small Lua script under resource.customizations in the argocd-cm ConfigMap — skip that step and a genuinely broken CRD-backed workload just sits at Healthy or Progressing forever, telling you nothing.

The Application CRD: the resource you actually write

☺ Like you're 10: One file says which Git folder to watch, which cluster and namespace it belongs in, and whether Argo CD is allowed to act on its own or has to wait for a person to say go.

An Application is the whole interface — a single custom resource, living in Argo CD's own namespace, that names a Git source, a destination, and a sync policy. Everything else in this page is really elaboration on one of those three fields.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: checkout
  namespace: argocd                            # Applications live in Argo CD's own namespace
  finalizers:
    - resources-finalizer.argocd.argoproj.io   # cascade: deleting this Application also prunes what it created
spec:
  project: checkout-team                       # NOT "default" in anything that matters — see AppProject below
  source:
    repoURL: https://github.com/acme/checkout-manifests.git
    targetRevision: main                       # a branch, a tag, or an exact commit SHA
    path: overlays/prod
    kustomize:
      images:
        - ghcr.io/acme/checkout=ghcr.io/acme/checkout:1.4.4
  destination:
    server: https://kubernetes.default.svc      # or `name:` for a cluster registered by name, not URL
    namespace: checkout
  syncPolicy:
    automated:
      prune: true          # delete resources removed from Git — OFF by default even under automated
      selfHeal: true        # revert live drift back to Git — also OFF by default
    syncOptions:
      - CreateNamespace=true
    retry:
      limit: 5
      backoff: { duration: 5s, factor: 2, maxDuration: 3m }
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/replicas         # an HPA owns this field now; stop reporting it as drift

source.targetRevision pinned to an exact commit SHA rather than a branch is how you get a fully immutable, auditable deploy — the same discipline Terraform's .terraform.lock.hcl enforces for providers, applied here to the whole manifest tree. Newer Argo CD releases also support a sources: list instead of a single source:, letting one Application combine a Helm chart from one repository with a values file from another — check the version your platform runs before assuming it's available.

Scaling out: app-of-apps and ApplicationSets

☺ Like you're 10: One Application can point at a whole folder of other Applications, and one template can stamp out a slightly different Application per cluster automatically.

One team, one service, one cluster — a single hand-written Application is plenty. Real platforms need dozens to thousands, and two composition patterns are how Argo CD scales without hand-writing every one.

The app-of-apps pattern is the simpler of the two: a root Application whose source.path points not at Kubernetes manifests but at a directory of other Application manifests. Sync the root, and Argo CD creates every child Application it finds there, each of which then reconciles independently against its own source and destination. It's the standard way to bootstrap a whole platform's baseline — ingress controller, Prometheus, cert-manager, and every application team's own root — from a single kubectl apply, and it's also how many teams manage Argo CD's own configuration: a root Application that points at Argo CD's own Helm values, so upgrading Argo CD is itself a GitOps-managed sync.

The ApplicationSet controller solves a related but distinct problem: not "many different apps," but "the same app, templated across many targets." A generator produces a list of parameter sets — List (a literal set you write out, as below), Cluster (one instance per cluster Argo CD has registered), Git (one instance per directory or file matched in a repo), Matrix (the cartesian product of two other generators — every app times every cluster), or Pull Request (one ephemeral instance per open PR, for preview environments) — and a template block turns each parameter set into a real Application, the same relationship a Helm chart has to its rendered manifests, just one level up the stack.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: checkout-environments
  namespace: argocd
spec:
  generators:
    - list:
        elements:
          - env: dev
            cluster: https://dev.k8s.acme.internal
          - env: staging
            cluster: https://staging.k8s.acme.internal
          - env: prod
            cluster: https://prod.k8s.acme.internal
  template:
    metadata:
      name: 'checkout-{{env}}'
    spec:
      project: checkout-team
      source:
        repoURL: https://github.com/acme/checkout-manifests.git
        targetRevision: main
        path: 'overlays/{{env}}'
      destination:
        server: '{{cluster}}'
        namespace: checkout
      syncPolicy:
        automated: { prune: true, selfHeal: true }

That one ApplicationSet replaces three hand-maintained Application files that would otherwise drift the moment someone edits one and forgets the other two — the exact copy-paste rot Helm's chart model exists to prevent, applied here to the deployment layer instead of the manifest layer.

Sync policies, waves, and hooks

☺ Like you're 10: You choose whether Argo CD acts on its own or waits for a person, you can force some pieces to go before others, and you can run one-off jobs right before or after a sync.

Without syncPolicy.automated, an Application is manual: Argo CD detects and reports OutOfSync forever but never applies anything until someone clicks Sync or runs argocd app sync. Set automated and reconciliation starts happening on its own — but prune and selfHeal are independent booleans, both defaulting to false even once automated is present, so an "automated" Application with neither flag set will apply new and changed resources on its own yet silently leave deleted-from-Git resources running forever and never correct a manual kubectl edit. All three flags need to be set deliberately for the loop this page describes to actually close.

Most objects in a sync apply together, but ordering sometimes matters — a namespace or a CRD genuinely has to exist before anything using it does. The argocd.argoproj.io/sync-wave annotation (default "0", any integer allowed) groups resources into waves applied lowest-first, and — critically — the controller waits for a wave to reach Healthy before starting the next one. A namespace at wave -1, application resources at wave 0, and a smoke-test Job at wave 1 is the common shape. Hooks are the finer-grained tool for one-off actions rather than ordering: a resource annotated argocd.argoproj.io/hook: PreSync (or Sync, PostSync, SyncFail) runs at that specific point in the sync rather than as a normal tracked resource, and hook-delete-policy: HookSucceeded,BeforeHookCreation controls whether the hook Job itself gets cleaned up automatically or piles up in the namespace after every deploy.

Day-to-day commands

☺ Like you're 10: Log in, point it at a repo and a cluster, and from there almost everything is list, look, diff, sync.

# first login — install generates a random admin password, stored as a Secret
$ kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d
$ argocd login argocd.acme.internal --username admin        # or --sso once Dex/OIDC is wired up

$ argocd repo add https://github.com/acme/checkout-manifests.git --ssh-private-key-path ~/.ssh/id_ed25519
$ argocd cluster add prod-cluster-context                   # registers a cluster from your local kubeconfig
$ argocd proj create checkout-team \
    --dest https://kubernetes.default.svc,'checkout*' \
    --src https://github.com/acme/checkout-manifests.git

$ argocd app create checkout \
    --repo https://github.com/acme/checkout-manifests.git \
    --path overlays/prod --revision main \
    --dest-server https://kubernetes.default.svc --dest-namespace checkout \
    --sync-policy automated --auto-prune --self-heal

$ argocd app list                                # every Application, sync status, health status
$ argocd app get checkout                        # full detail: source, destination, resource tree
$ argocd app diff checkout                       # Git desired state vs. live cluster state, right now
$ argocd app sync checkout                       # force a sync outside the reconciliation loop
$ argocd app sync checkout --prune               # include deletions this run, even if prune is off
$ argocd app history checkout                    # every synced revision, oldest to newest
$ argocd app rollback checkout 7                 # re-apply a PRIOR revision — see the warning above
$ argocd app wait checkout --health --timeout 300   # block, e.g. in CI, until Healthy or timeout
$ argocd app set checkout --sync-policy automated   # flip a manually-managed app to automated

Gotchas and failure modes

☺ Like you're 10: Most surprises come from a flag you thought was on being off by default, or from Git and the cluster disagreeing about what "current" means.

Non-deterministic charts turn into a permanent, unresolvable diff

Helm's own gotchas already warn that charts using randAlphaNum, now, or lookup render differently on every single pass. Under a push-based pipeline that's a one-time nuisance; under Argo CD's continuous reconciliation it's permanent, meaningless OutOfSync that never resolves, because the repo server re-renders the chart fresh on every diff and gets a slightly different answer every time. There's no fix inside Argo CD for this — it has to be fixed in the chart, by pinning the value that function was generating (a real Secret, a fixed image digest) instead of generating it at render time.

Cascading prune: deleting a parent can delete everything under it

The resources-finalizer.argocd.argoproj.io finalizer shown in the Application example above is what makes prune: true genuinely dangerous to misjudge on an app-of-apps root. Delete a root Application carrying that finalizer, and Argo CD doesn't just remove the Application object — it prunes every resource that Application (and, transitively, every child Application it created) is tracking, cascading all the way down. kubectl delete application root-app --cascade=false (or the finalizer's absence) deletes only the Application object and leaves everything it created running, which is almost always what you actually want when the thing that's broken is the Application's own config, not the workloads underneath it.

An unscoped project trusts every repo and every destination

The default AppProject that ships out of the box allows any source repository and any destination cluster and namespace — fine for a demo, a real security gap in production, where it means any Application in that project can point at any Git repo and deploy to any namespace Argo CD can reach. An AppProject scopes that down explicitly:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: checkout-team
  namespace: argocd
spec:
  sourceRepos:
    - https://github.com/acme/checkout-manifests.git
  destinations:
    - server: https://kubernetes.default.svc
      namespace: 'checkout*'
  clusterResourceWhitelist: []              # no cluster-scoped resources — namespaces, CRDs stay off-limits
  namespaceResourceBlacklist:
    - group: ''
      kind: ResourceQuota

Every Application's spec.project field points at one of these — never leave it defaulted to default on anything that matters, the same way you wouldn't leave a cloud IAM role unscoped just because the console offered a working default.

Argo CD vs. the alternatives

☺ Like you're 10: Other tools reconcile or deploy the same kind of thing — they just trade a built-in dashboard, a lighter footprint, or reach beyond Kubernetes differently.

The practical rule: reach for a GitOps reconciler once "who deployed this, from where, and can we prove it" needs a definitive answer, and once drift correction matters more than deploy simplicity — Argo CD if a built-in UI and easy multi-cluster fan-out matter, Flux if a lighter footprint and tighter Kubernetes-native tooling matter more. Argo CD doesn't appear as a named domain on the AWS DevOps Engineer – Professional (DOP-C02) exam this course's certifications page centers on — see Resilient Cloud Solutions for the AWS-native side of this same territory — but the GitOps pattern itself is explicit, named content on KCNA's curriculum alongside the wider CNCF landscape; confirm the current exam guide before betting study time on either one.