Kubernetes

Kubernetes is the control plane that takes a declarative description of what you want running — this many replicas of this image, reachable at this address — and continuously drives the cluster's actual state to match it, correcting on its own whenever a Pod crashes, a node dies, or someone edits something by hand. Containers & orchestration already named Pods, Deployments, and Services in passing and introduced the shape of that control loop; this page goes deeper into all three, into the reconciliation model that makes them self-healing, and into the actual YAML and kubectl commands you need to ship a workload onto a cluster with confidence. It deliberately does not go as deep as running one: cluster bootstrapping, etcd backup and restore, RBAC design, CNI networking internals, storage provisioning, and version upgrades are cluster-administrator territory, covered end to end on the CKA certification profile. This page is written at the altitude a DevOps engineer actually operates at — someone who deploys onto a cluster someone else keeps alive, not someone who built the cluster.

What a DevOps engineer needs here, and what belongs to CKA

☺ Like you're 10: You need to know how to move into the apartment and live there comfortably — you don't need to know how the building's plumbing was installed.

Kubernetes has one API surface but two very different jobs sitting behind it, and conflating them is the single most common reason people bounce off this page's subject feeling like they need a semester before they can ship anything. The cluster administrator's job is keeping the cluster itself alive: installing and upgrading the control plane, managing etcd, wiring up the container network interface (CNI) and storage classes, writing RBAC policy, patching nodes. The DevOps engineer's job — the one this page covers — is using a cluster that already exists to run workloads reliably: writing correct manifests, reading what the cluster is telling you when something's wrong, and knowing which command fixes which failure. You can do the second job extremely well without ever having run kubeadm init.

If you're the one who also owns the cluster, or you're studying for the exam that certifies that role, everything in the right-hand column above gets full treatment on the CKA — Certified Kubernetes Administrator profile. If your interest is specifically writing and deploying application manifests — a slightly different exam with a slightly different emphasis — see CKAD instead; it maps onto this page's scope more closely than CKA does. Either way, verify current exam format, domain weights, and pricing on the CNCF's own pages before you commit study time — certification details change more often than the underlying technology does.

The control plane and the reconciliation loop

☺ Like you're 10: A handful of specialist workers each do one job — one writes down what you asked for, one decides which machine runs what, one actually starts it — and they never stop checking their own piece of the promise.

Every Kubernetes cluster splits into two kinds of machines. The control plane makes decisions and never runs your application containers; worker nodes run nothing but your workloads. What "Kubernetes" actually is, mechanically, is a small set of independent processes that only talk to each other through one shared, versioned record of desired and observed state.

• kube-apiserver — the front door and the only thing anything else talks to. It validates every request, is the sole reader and writer of etcd (the cluster's key-value store, and the only place state actually lives), and exposes a watch API so every other component can subscribe to changes instead of polling.
• etcd — a distributed, strongly consistent key-value store holding the entire cluster's state. Nothing else in the cluster reads or writes it directly; everything goes through the API server. Backing it up and recovering it is a cluster-administrator skill covered on the CKA page, not here.
• kube-scheduler — watches the API server for Pods that exist but haven't been assigned to a Node yet, scores every candidate Node against the Pod's resource requests and any placement rules, and writes a binding back through the API server.
• kube-controller-manager — runs dozens of independent control loops bundled into one process, each responsible for one object type. The Deployment controller watches Deployments and creates or updates ReplicaSets to match; the ReplicaSet controller watches ReplicaSets and creates or deletes Pods to match the replica count. Each loop only knows about its own object type.
• kubelet — an agent running on every worker Node, watching the API server for Pods assigned to its own Node, and talking to a container runtime (containerd, via the Container Runtime Interface) to actually pull images and start containers. It reports Pod and Node status back up.
• kube-proxy — runs on every Node and implements Service networking, programming rules (via iptables or IPVS) so traffic sent to a Service's stable address gets routed to one of the currently-healthy Pods behind it.

The core objects: Pod, Deployment, Service

☺ Like you're 10: A Pod holds your container, a Deployment keeps the right number of Pods running and rolls out new versions, and a Service hands out one steady address so nobody has to track which Pod moved where.

Pod is the smallest deployable unit — a wrapper around one or more containers sharing one network namespace (one IP) and, optionally, storage volumes. You almost never write a bare Pod for a real workload: Pods are disposable by design, and something needs to notice when one dies and create a replacement. That something is a Deployment, which owns a ReplicaSet, which owns the Pods — a Deployment you write directly, a ReplicaSet you almost never touch by hand, and Pods you mostly only read. A Service then gives that shifting set of Pods one address that doesn't change as individual Pods come and go.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout-api
  labels: { app: checkout-api }
spec:
  replicas: 3
  selector:
    matchLabels: { app: checkout-api }        # MUST match template.metadata.labels below
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1          # up to 1 extra Pod above `replicas` during a rollout
      maxUnavailable: 0    # never drop below `replicas` Ready Pods while rolling
  template:
    metadata:
      labels: { app: checkout-api }
    spec:
      containers:
        - name: checkout-api
          image: registry.internal/checkout-api:1.4.2   # pin a digest or a never-reused tag in prod
          ports:
            - containerPort: 3000
          env:
            - name: LOG_LEVEL
              valueFrom:
                configMapKeyRef: { name: checkout-config, key: logLevel }
          resources:
            requests: { cpu: 100m, memory: 128Mi }   # what the scheduler reserves for this Pod
            limits:   { memory: 256Mi }               # exceed this and the container is OOMKilled
          readinessProbe:                             # fail this → removed from Service, NOT restarted
            httpGet: { path: /healthz, port: 3000 }
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:                              # fail this → container is killed and restarted
            httpGet: { path: /healthz, port: 3000 }
            initialDelaySeconds: 15
            periodSeconds: 20
---
# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: checkout-api
spec:
  selector: { app: checkout-api }    # every label here must match a Pod's labels, or Endpoints is empty
  ports:
    - port: 80
      targetPort: 3000
  type: ClusterIP                    # NodePort or LoadBalancer expose it beyond the cluster

Two fields are worth reading twice because they're the source of most first-week mistakes. A Deployment's spec.selector.matchLabels must match its own spec.template.metadata.labels — Kubernetes rejects an update that changes the selector on an existing Deployment, because changing what a Deployment considers "its" Pods retroactively is not a safe operation. And a Service's spec.selector is matched against Pod labels independently, by string equality — it has no relationship to the Deployment at all beyond both happening to use the same labels; a Service will happily route to Pods from three different Deployments, or to zero Pods, based purely on whether labels line up.

Beyond these three, a DevOps engineer runs into a handful of other objects constantly and a longer tail rarely — the constant ones are worth naming even though most get their own fuller treatment elsewhere in this course:

Day-to-day kubectl commands

☺ Like you're 10: A handful of commands cover almost everything: ship it, look at it, read what it's saying, and undo it if it's wrong.

# ship and inspect
$ kubectl apply -f deployment.yaml -f service.yaml   # create or update — idempotent, use this over `create`
$ kubectl get deployments,pods,svc -n checkout -o wide
$ kubectl describe pod checkout-api-7d8f9c-xk2p9      # the single most useful triage command — read the Events at the bottom
$ kubectl get endpoints checkout-api                  # empty list = Service selector matches no Pod, full stop

# logs and a shell
$ kubectl logs -f checkout-api-7d8f9c-xk2p9           # follow live logs
$ kubectl logs checkout-api-7d8f9c-xk2p9 --previous   # logs from the container BEFORE its last restart — the CrashLoopBackOff command
$ kubectl exec -it checkout-api-7d8f9c-xk2p9 -- sh    # a shell inside the running container

# rollouts
$ kubectl rollout status deployment/checkout-api      # blocks until the rollout finishes or fails
$ kubectl rollout history deployment/checkout-api     # every revision, with --revision=N for detail
$ kubectl rollout undo deployment/checkout-api         # back to the previous revision
$ kubectl rollout undo deployment/checkout-api --to-revision=3

# scale, debug, and clean up
$ kubectl scale deployment/checkout-api --replicas=5
$ kubectl top pods -n checkout                        # live CPU/memory, needs metrics-server installed
$ kubectl get events -n checkout --sort-by=.lastTimestamp   # cluster-wide, chronological — the first thing to check
$ kubectl port-forward svc/checkout-api 8080:80        # reach a ClusterIP Service from your laptop, no Ingress needed
$ kubectl delete pod checkout-api-7d8f9c-xk2p9         # deletes ONE Pod; the ReplicaSet immediately creates a replacement
$ kubectl explain deployment.spec.strategy.rollingUpdate  # field-level docs from the API itself — works offline

kubectl delete pod is worth calling out on its own: deleting a Pod that a ReplicaSet owns doesn't remove capacity, it just forces a fresh one — the fastest "have you tried turning it off and on again" available, and safe precisely because the reconciliation loop notices the gap and closes it within seconds.

Gotchas and failure modes

☺ Like you're 10: Almost every real incident is one of a handful of shapes — a container that won't stay up, a Pod nobody will run, or a Service pointing at nothing.

CrashLoopBackOff and ImagePullBackOff

CrashLoopBackOff means the container starts and exits immediately, repeatedly, and the kubelet is backing off between restart attempts (capped around five minutes). It's almost always an application-level problem — a missing environment variable, a bad startup command, a dependency that isn't reachable yet — not a Kubernetes problem, and kubectl logs --previous is the fastest way to see the exit before the next restart wipes it. ImagePullBackOff / ErrImagePull means the kubelet can't fetch the image at all: a typo'd tag, an image that was never pushed, or a private registry the Node has no imagePullSecret for. kubectl describe pod's Events section names the exact reason for both.

Pending: the Pod the scheduler won't place

A Pod stuck Pending has never been scheduled to a Node at all — no container has started, because the scheduler couldn't find a Node that satisfies it. The three usual causes are insufficient CPU or memory across every Node relative to the Pod's resources.requests, a nodeSelector or affinity rule that no Node's labels satisfy, and a taint on every eligible Node with no matching toleration on the Pod. kubectl describe pod again has the answer, in a line reading something like "0/6 nodes are available: 6 Insufficient memory."

Readiness vs. liveness: two probes, two very different consequences

A failing readinessProbe removes the Pod from every Service's Endpoints — traffic stops routing to it — but the container keeps running untouched. A failing livenessProbe kills the container and restarts it. Confuse the two, or set a livenessProbe's initialDelaySeconds too aggressively for a slow-starting app, and you get a container killed mid-boot in a loop that looks exactly like CrashLoopBackOff but is actually self-inflicted by the probe config — a startupProbe exists specifically to give slow-starting containers room before liveness checks begin. The opposite failure is quieter and more dangerous: a readinessProbe that fails forever due to a misconfigured path leaves a Pod running, consuming resources, showing as healthy in most dashboards, and permanently receiving zero traffic — kubectl get endpoints is what actually surfaces it.

OOMKilled, throttling, and skipping requests/limits entirely

Exceed a container's memory limit and the kernel kills it (SIGKILL, reported as OOMKilled in kubectl describe pod) — memory can't be throttled the way CPU can, only reclaimed by force. Exceed a CPU limit and the container is throttled, not killed, which shows up as mysterious latency rather than a restart. Skip requests entirely and the Pod gets Kubernetes' BestEffort quality-of-service class — first in line for eviction the moment its Node comes under memory pressure, regardless of how much memory the container is actually using.

A rollout that hangs forever

If new Pods from a rollout never pass their readinessProbe — a bad image, a config error only the new version hits — a rolling update with the default maxUnavailable: 25% just stalls: old Pods keep serving, new ones sit un-Ready, and kubectl rollout status never returns. Nothing is actually broken from a user's perspective yet, which is exactly why it's easy to miss; kubectl rollout undo is the fix, and practicing the diagnosis-to-rollback path on a real stuck rollout is the whole point of Drill — Roll Back a Bad Deploy.

Manual edits get reconciled away

kubectl edit deployment/checkout-api or a hand-patched live object works — for exactly as long as nothing re-applies the manifest that doesn't match your edit. Under GitOps with Argo CD that can be minutes; even without GitOps, the next ordinary kubectl apply from CI silently overwrites the hand edit. Fix the source manifest and let it flow through the pipeline, the same discipline covered generally in infrastructure as code — a live cluster edit is a debugging tool, never a fix.

Kubernetes vs. its alternatives

☺ Like you're 10: Other tools solve the same "keep my containers running" problem with a smaller rulebook — the trade is almost always less power for less to learn.

The practical rule most teams land on: reach for a managed Kubernetes offering (EKS/GKE/AKS) once you need portability, a mature ecosystem, or genuine multi-service scale, and reach for something narrower — ECS, Cloud Run, Nomad — when a single cloud and a simpler operational model outweigh Kubernetes' extensibility. Very few teams that adopt Kubernetes for a handful of services actually needed its full feature surface on day one; see the DevOps toolchain for where it sits relative to everything else on a pipeline.

Going further: CKA, CKAD, and the rest of the certification track

☺ Like you're 10: This page gets you deploying confidently — three deeper exams exist for the people who own the cluster, build apps on it full-time, or secure it.

Everything above is deliberately the DevOps-engineer slice of a much larger subject. Three vendor-neutral certifications from the Cloud Native Computing Foundation cover the deeper layers, each with a different center of gravity, and it's worth knowing which one actually matches what you're trying to get better at before you commit study hours to any of them:

• CKA — Certified Kubernetes Administrator — the cluster-operator exam: installation, upgrades, etcd, RBAC, networking, storage, and troubleshooting a cluster itself. Everything this page waved past as "out of scope" lives here.
• CKAD — Certified Kubernetes Application Developer — the closest match to this page's actual scope: designing, building, and deploying applications onto a cluster someone else administers. If this page felt like the right altitude, CKAD is the exam that certifies it.
• CKS — Certified Kubernetes Security Specialist — a CKA prerequisite, focused specifically on cluster and workload hardening.

Also worth knowing about: KCNA — Kubernetes and Cloud Native Associate is a lighter-weight, no-prerequisite entry point across the whole cloud-native landscape, not Kubernetes hands-on depth specifically — a reasonable starting point if all three exams above feel premature. All four exam formats, current pricing, and domain weightings change more often than the underlying platform does; verify specifics on the CNCF's own certification pages before scheduling. For the canonical technical reference behind everything on this page, kubernetes.io/docs is the primary source, and it's worth reading directly rather than through a summary once you're past this introduction. Practice the objects and commands on this page against a real (if small) deployment in Capstone Part 3 — Deployment Strategy, and see how a chart packages the manifests above for reuse in Helm.