Multi-Cluster & Fleet Management

Everything the CKA, CKAD, and CKS blueprints test assumes one cluster: one API server to talk to, one etcd quorum underneath, one CNI, one set of nodes. That assumption is correct for a huge share of real workloads and wrong for the rest — sooner or later a regulator asks where the data physically sits, a bad upgrade takes down every workload at once, or a team needs an isolated blast radius. The moment a second cluster exists, a new problem appears that no single cluster ever poses: keeping many clusters built the same way, configured the same way, and reachable from each other, instead of each one quietly drifting into its own snowflake. This page is the Kubernetes-native tour of that problem — why teams split in the first place, how Cluster API turns "build a cluster" into an object a controller reconciles instead of a hand-run script, how GitOps and policy get pushed to an entire fleet at once, and how a Service in one cluster becomes resolvable from another through the Multi-Cluster Services API. None of it is exam-blueprint material; all of it is what "production Kubernetes" means past a certain scale.

Why one cluster stops being enough

☺ Like you're 10: One fire station is simple to run but risky — if it has a bad day, the whole town has no cover. Splitting into a few gives every neighborhood its own safety net.

Kubernetes Architecture and Control Plane Internals both describe one cluster as a single, cohesive unit: one API server surface, one etcd Raft quorum, one CNI, one admission chain. That cohesion is also the risk — a Kubernetes cluster is a shared-fate unit. A botched control-plane upgrade, a runaway admission webhook denying every request, or an etcd disk filling up degrades every workload on that cluster simultaneously, no matter how unrelated they are. Splitting workloads across clusters turns one catastrophic outage into several smaller, independent ones, and it's the single most common honest reason a team runs more than one cluster: blast-radius isolation.

Three more forces show up just as often, and each implies a different split. Hard tenancy: namespaces, RBAC, and NetworkPolicies give real but soft isolation inside one cluster — tenants still share one API server and one kernel per node — so mutually distrusting parties (different SaaS customers, PCI-scoped workloads next to general ones) need a genuinely separate cluster, not just a separate namespace. Data residency and compliance: regulation like GDPR or India's DPDP can require that a region's data never leaves it and that no operator outside that jurisdiction can touch it, which a single global control plane simply cannot promise. Latency, disaster recovery, and the scale ceiling: users on another continent pay real round-trip latency to a single distant cluster, surviving a regional outage means a second region already running rather than one improvised afterward, and every control plane has a real capacity ceiling — you'll feel etcd and API-server pressure well before you hit it.

Cluster API: making the cluster itself a reconciled object

☺ Like you're 10: Instead of a different crew building each fire station by hand and forgetting how, you write the blueprint down once and a machine builds every station from it — and quietly rebuilds any wall that starts to crack.

Hand-building clusters — clicking through a cloud console, or running a slightly different script each time — is exactly how a fleet accumulates snowflakes: every cluster subtly different, no two upgrades alike, nobody able to recreate one from scratch. Cluster API (CAPI), a Kubernetes SIG Cluster Lifecycle project, fixes this by doing to clusters exactly what the object model already does to workloads: you declare desired state as ordinary Kubernetes objects, and a controller reconciles reality toward it, forever. A small management cluster runs the CAPI controllers; the workload clusters it creates are the ones that actually run your Pods, and ideally know nothing about CAPI at all. Because CAPI objects are just YAML, they live in Git and get applied by the same GitOps controller covered two sections down — "stand up a cluster in ap-south" becomes a reviewed pull request, not a person with cloud-console access at 2 a.m.

CAPI splits cleanly into three provider roles that compose to build one node: an infrastructure provider (CAPA for AWS, CAPZ for Azure, CAPD for local Docker-based clusters) that creates the real VMs, networks, and load balancers; a bootstrap provider — almost always Kubeadm — that turns a raw machine into a node by generating its cloud-init; and a control-plane provider (KubeadmControlPlane) that owns the API-server/etcd tier as a unit, including its own HA and rolling upgrades. Worker nodes are grouped into a MachineDeployment, and the symmetry with what you already know is the entire design: MachineDeployment → MachineSet → Machine is the exact same ownership shape as Deployment → ReplicaSet → Pod, and changing a version field triggers the same kind of rolling replacement a Deployment does — except what gets replaced is a whole node, never patched in place.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: prod-ap-south
  namespace: fleet
spec:
  clusterNetwork:
    pods: { cidrBlocks: ["10.244.0.0/16"] }      # must not overlap any other cluster in the fleet
  controlPlaneRef:                                # the control-plane provider
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: prod-ap-south-cp
  infrastructureRef:                              # the infrastructure provider
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
    kind: AWSCluster
    name: prod-ap-south
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment                           # a worker pool — clusters' own "Deployment"
metadata:
  name: prod-ap-south-md-0
  namespace: fleet
spec:
  clusterName: prod-ap-south
  replicas: 6                                     # scale the pool like any Deployment
  template:
    spec:
      clusterName: prod-ap-south
      version: v1.30.2                            # bump this → CAPI rolls new, never-patched nodes
      bootstrap:
        configRef: { apiVersion: bootstrap.cluster.x-k8s.io/v1beta1, kind: KubeadmConfigTemplate, name: prod-ap-south-md-0 }
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
        kind: AWSMachineTemplate
        name: prod-ap-south-md-0

This page's job is the concept, not the command line — the full clusterctl workflow, ClusterClass managed topologies, and the failure modes that bite real fleets (provider version skew, a MachineHealthCheck remediation loop shredding a whole pool, the "pivot" that moves CAPI itself between management clusters) all live in Platform Engineering's Cluster API tool guide, written for exactly this depth. What matters here is the shape of the idea: a cluster is no longer a thing a person builds once and remembers — it's a reconciled object, the same way a Pod is, and that's what makes everything in the rest of this page possible.

Fleet-wide GitOps: the same config, pushed to every cluster

☺ Like you're 10: You wrote one rulebook. Now every fire station needs the same rulebook, automatically, with only the small local tweaks each one actually needs.

Cluster API births an empty cluster — a working control plane and joined nodes, nothing else. Something has to fill it with the actual platform (ingress, cert-manager, monitoring agents, tenant namespaces) and keep it in step with every other cluster forever, and that's GitOps stretched across a fleet instead of one cluster. The dominant pattern is a single Argo CD running on the hub with an ApplicationSet driven by the cluster generator: every workload cluster is registered with labels like region=ap-south, the generator produces one Application per matching cluster from a single template, and those label values flow straight into the template so each cluster pulls its own region-specific overlay. Register a brand-new cluster and the platform's whole add-on stack reconciles onto it — no one writes a new manifest.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: platform-addons
  namespace: argocd
spec:
  generators:
    - clusters: { selector: { matchLabels: { fleet: "true" } } }   # one App per registered cluster
  template:
    metadata: { name: 'addons-{{name}}' }
    spec:
      source:
        repoURL: https://git.example.com/fleet-config.git
        path: 'addons/overlays/{{metadata.labels.region}}'         # per-region overlay
      destination: { server: '{{server}}', namespace: platform-system }
      syncPolicy: { automated: { prune: true, selfHeal: true } }

Argo CD isn't the only shape this takes — Flux favors a reconciler running inside each cluster pulling the same repo, which scales further and survives a partition at the cost of no single pane of glass, while Rancher Fleet and Sveltos trade off differently again on where the reconciler runs and how rich the cluster-targeting rules can be. The full engine-by-engine comparison, with a working table of trade-offs, is in Platform Engineering's Multi-Cluster & Fleet Management — the point that matters for a Kubernetes-focused reader is simpler: whichever engine you pick, the same config must reach every cluster from the same source, or the fleet's guardrails start drifting.

And guardrails are exactly where a fleet's real risk lives. Admission policy — the same rules RBAC & Admission Control covers for one cluster — is only as strong as its coverage across all of them: ship the identical policy bundle to every cluster as just another add-on in the same GitOps stream, so "no privileged Pods" and "images only from our registry" hold everywhere by construction, not by memory. Identity should federate the same way — every cluster's API server pointed at the same OIDC provider, so a person's group membership maps to identical RBAC everywhere, provisioned by GitOps rather than hand-edited per cluster. The anti-pattern to watch for is a per-cluster local-admin account or a hand-crafted kubeconfig: invisible, prone to drift, and exactly how a fleet accumulates a snowflake with god-mode access. Platform Engineering's governance & compliance covers the fleet-wide policy-reporting side of this in full.

Service discovery across clusters: the Multi-Cluster Services API

☺ Like you're 10: Each fire station has its own phone book. If someone at one station needs to call a specific person at another, both stations need to share phone books first — and the phones need a wire between the buildings before any call goes through.

Networking & the CNI covers how, inside one cluster, a Service name resolves through CoreDNS and packets reach any Pod over a flat, no-NAT network. Neither of those things is true by default the instant a second cluster enters the picture: each cluster has its own CoreDNS, its own Pod CIDR, and no route at all to any other cluster's Pods. Closing that gap takes two independent layers — a data plane that makes cross-cluster Pod-to-Pod traffic actually route (and stay encrypted), and a discovery plane that lets a name resolved in one cluster point at endpoints living in another. Kubernetes standardizes the second layer as the Multi-Cluster Services (MCS) API, a SIG-Multicluster project.

You group member clusters into a ClusterSet, then use two CRDs: a ServiceExport, created in the cluster that owns the Service, marks it shareable with the rest of the set; the MCS implementation then auto-creates a matching ServiceImport in every other member. An exported Service becomes resolvable at a well-known name — <service>.<namespace>.svc.clusterset.local — and, crucially, endpoints from every exporting cluster are merged behind that one name, so it naturally load-balances and fails over across clusters without any application code knowing which cluster actually answered.

# In the exporting cluster (ap-south): mark the Service shareable with the ClusterSet
apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceExport
metadata:
  name: catalog
  namespace: shop
---
# From any member cluster (including eu-west), the name now resolves fleet-wide:
#   catalog.shop.svc.clusterset.local  →  merged endpoints from every exporting cluster
# kubectl --context eu-west run probe --rm -it --image=busybox -- \
#   nslookup catalog.shop.svc.clusterset.local

There's a hard prerequisite underneath all of it: Pod and Service CIDRs must not overlap across any two clusters in the set. If ap-south and eu-west both hand out addresses from 10.244.0.0/16, a cross-cluster route is genuinely ambiguous — the exact reason the Cluster API example two sections up pins each cluster's cidrBlocks explicitly. Plan the fleet's address space before the second cluster exists, not after. MCS itself is a vendor-neutral contract, not an implementation — the actual cross-cluster routing underneath it is provided by something else: Cilium Cluster Mesh connects clusters at the CNI/eBPF layer with transparently encrypted (WireGuard or IPsec) traffic, while Istio or Linkerd multi-cluster extends mTLS and traffic policy across clusters via east-west gateways — the mesh side of that trade-off is Service Mesh Fundamentals' territory, and the fleet-scale version of the whole discussion, including global load balancing for user-facing failover, is in Platform Engineering's Multi-Cluster & Fleet Management.

When one more cluster isn't the answer

☺ Like you're 10: A new fire station isn't free — someone has to staff it, train it, and inspect it forever. Sometimes a bigger room in the existing station really is enough.

Everything above assumes the extra cluster is justified — but every cluster is a standing bill, paid in managed control-plane fees, per-node overhead for its own CNI and log agents, and a permanent place in on-call. Before reaching for a whole new cluster, escalate only as far as the actual driver demands: a well-configured namespace (quotas, NetworkPolicy, RBAC) handles most team isolation between parties that trust each other; a virtual cluster gives a team its own API server and CRDs inside a shared host cluster, most of the isolation at a fraction of the tax; a real separate cluster is for genuinely distrusting tenants or a hard compliance boundary; a separate region is for latency, DR, or data residency law. Adding a cluster when a namespace would do is the same over-engineering trap covered in Anti-Patterns & Pitfalls; refusing to add one when residency law genuinely demands it is the opposite mistake. Platform Engineering's Multi-Cluster & Fleet Management walks the full cost tax — including the N-clusters × M-add-ons multiplication that makes fleet automation non-optional past a handful of clusters — in detail this page deliberately doesn't repeat. If your own path runs through the full CNCF ladder rather than just Kubernetes' own certifications, the Golden Astronaut course covers the other CNCF projects and LFCS on the way to Golden Kubestronaut.