Interview Q&A Reference

🟡 Q1 · Walk me through kubectl apply

As they'll ask it: "I run kubectl apply -f deployment.yaml. Walk me through everything that happens before my Pods are actually running."

Model answer. kubectl reads the YAML, works out a three-way merge against the object's last-applied state (or uses server-side apply's field-manager tracking), and sends an HTTP request to kube-apiserver. The apiserver runs it through authentication, then authorization (an RBAC check — does this identity have create/patch on Deployments here?), then the admission chain: mutating webhooks first, which can modify the object, then validating webhooks and built-in checks, which can only accept or reject. Once admitted, the apiserver persists the object to etcd — the only component that talks to etcd directly.

From there it's controllers reacting to what they're watching, not a single chain of calls. The Deployment controller notices the new Deployment and creates a ReplicaSet; the ReplicaSet controller creates Pod objects with no nodeName yet. kube-scheduler watches for exactly that — unscheduled Pods — filters and scores the nodes, and writes a Binding back through the apiserver. The kubelet on that node, watching for Pods bound to itself, pulls the image via the container runtime over the CRI, starts the container, and reports status back up through the apiserver into etcd. Every one of those arrows is a watch on the apiserver, not a direct call between components.

What they're really checking: whether you think of Kubernetes as a synchronous pipeline or as a set of independent controllers converging on shared state. The second model is the correct one, and it's what the rest of the interview builds on.

🟢 Q2 · Pod vs ReplicaSet vs Deployment

As they'll ask it: "What's the actual difference between a Pod, a ReplicaSet, and a Deployment? Why do we need all three?"

Model answer. A Pod is the smallest deployable unit — one or more containers sharing a network namespace and IPC — and on its own it has no self-healing: if it dies and nothing else is watching, it stays dead. A ReplicaSet fixes that by continuously reconciling "how many Pods matching this selector exist" against a desired replica count, recreating any that disappear — but it has no concept of a rollout; changing its Pod template doesn't update existing Pods.

A Deployment sits above the ReplicaSet and owns the rollout: it creates a new ReplicaSet per template change, scales the new one up and the old one down according to maxSurge/maxUnavailable, keeps the old ReplicaSets around (scaled to zero) so kubectl rollout undo has something to restore, and tracks revision history. Three layers because each one solves a different problem: the Pod is the unit of execution, the ReplicaSet is the unit of self-healing, and the Deployment is the unit of change management.

What they're really checking: baseline fluency with the object model before anything else gets asked. A shaky answer here colours the rest of the round.

Common wrong answer: "A Deployment is just a fancier Pod." It skips the ReplicaSet entirely and misses that self-healing and rollout management are two separate jobs, done by two separate controllers.

🟡 Q3 · etcd's role, and losing quorum

As they'll ask it: "What does etcd actually do in a cluster, and what happens if it loses quorum?"

Model answer. etcd is a distributed, strongly consistent key-value store using the Raft consensus algorithm, and it's the sole source of truth for cluster state — every object, every status field, all of it. Only kube-apiserver talks to it directly; no other component, not even the scheduler or kubelet, reads or writes etcd itself. That's a deliberate least-privilege boundary, not an implementation detail.

Raft needs a majority of members alive to accept writes, which is why production etcd runs with an odd count — 3 or 5 — so a majority is unambiguous. Lose quorum (2 of 3 members down) and etcd stops accepting writes: the apiserver can still serve reads from its watch cache for a while, and Pods already running keep running because kubelet doesn't need etcd for Pods it already knows about — but nothing new happens. No scheduling, no new objects, no reconciliation of anything not already in flight. Recovery means restoring quorum or restoring from a snapshot (etcdctl snapshot save, taken on a schedule, is the actual backup story — this is a separate discipline from an application-level backup tool like Velero, which backs up Kubernetes objects via the API, not etcd's raft log).

What they're really checking: whether you understand that etcd availability, not apiserver availability, is usually the real single point of failure in a self-managed control plane — and whether "we run 3 etcd nodes" is something you understand or something you've just heard.

🟡 Q4 · How the scheduler picks a node

As they'll ask it: "How does the scheduler actually decide which node a Pod lands on?"

Model answer. Two phases. Filtering first: throw out every node that's infeasible — not enough allocatable CPU/memory for the Pod's requests, a taint the Pod doesn't tolerate, a nodeSelector or node affinity that isn't satisfied, a hard pod anti-affinity that would be violated, a hostPort conflict, a volume that can't attach from that node's topology. What's left is the feasible set, and it can legitimately be empty — that's a Pending Pod with a FailedScheduling event.

Scoring second: rank the feasible nodes — spreading Pods to balance resource utilisation, preferring nodes that already have the container image cached, honouring soft (preferred) affinity/anti-affinity rules, applying topology spread constraints. Highest score wins, and the scheduler writes a Binding object back through the apiserver, which is what actually sets spec.nodeName. This whole thing is pluggable — the "Scheduling Framework" exposes extension points (PreFilter, Filter, Score, Reserve, Permit, Bind) so a custom scheduler or plugin can hook in without forking the binary, and a Pod can opt into a non-default scheduler entirely via spec.schedulerName.

What they're really checking: whether "the scheduler picks the best node" is a black box to you or a two-phase algorithm you can actually describe — filter, then score, are two different jobs with two different failure modes.

🔴 Q5 · Level-triggered vs edge-triggered reconciliation

As they'll ask it: "Kubernetes controllers are described as 'level-triggered.' What does that mean, and why does it matter?"

Model answer. A level-triggered controller doesn't react to an event — "a Pod was just created" — it reacts to a state: on every reconcile tick it reads the current desired spec and the current observed status, computes the diff, and takes whatever action closes that diff, regardless of what specific change triggered the tick or whether it even saw that change at all. Compare that to edge-triggered systems, which fire once on a specific transition — a webhook call on "order placed" — and if that one call is dropped, the system never finds out on its own.

This is why Kubernetes is built around it: a controller can crash, restart, or simply miss a watch event during a network blip, and it doesn't matter, because the next reconcile just re-reads the actual current world state and converges toward the spec regardless of history. There's no event log to replay and no missed-webhook failure mode. The controller pattern generalises this everywhere — a Deployment's spec is desired state, its status is observed state, and every controller in the cluster, including every custom operator, is running the same "observe, diff, act" loop against its own pair of fields. See the Kubernetes API & the controller pattern for where this idea originates and operators & CRDs for where it gets extended to custom resources.

What they're really checking: senior signal. Most candidates can name "the reconciliation loop"; fewer can explain why it's specifically resilient to crashes and dropped events, which is the actual engineering reason it was chosen.

🟢 Q6 · A Pod is stuck Pending

As they'll ask it: "A Pod's been sitting in Pending for ten minutes. What do you check?"

Model answer. Pending specifically means the scheduler hasn't been able to bind it to a node yet, so my first move is always kubectl describe pod <name> and read the Events block — a Pending Pod almost always has a FailedScheduling event with a plain-English reason attached, and guessing before reading it wastes time. Common causes, roughly in the order I'd check them: no node has enough allocatable CPU or memory for the requested amounts (check with kubectl describe node, comparing Allocatable against Allocated resources); a taint on every candidate node with no matching toleration on the Pod; a nodeSelector or affinity rule no node satisfies; a PersistentVolumeClaim that hasn't bound yet, which blocks scheduling entirely if the volume mode requires it; or a namespace ResourceQuota already exhausted.

The event message almost always names which of these it is directly, so the actual skill here is reading it rather than diagnosing blind.

What they're really checking: whether "describe, then read" is your actual first reflex, or whether you start guessing at causes before looking at the one place Kubernetes already told you the answer.

🟢 Q7 · CrashLoopBackOff

As they'll ask it: "A Pod's in CrashLoopBackOff. Walk me through debugging it."

Model answer. CrashLoopBackOff means the container starts, exits, and kubelet is restarting it with an exponential backoff — 10s, 20s, 40s, up toward a cap around five minutes — not that anything is stuck. The first thing I check is the exit code, from kubectl describe pod's State: Terminated block: 137 is SIGKILL, almost always an OOMKill from exceeding the memory limit; a small nonzero code is usually the application itself exiting on a fatal error — bad config, a missing environment variable or Secret key, a dependency it can't reach with no retry logic.

Then kubectl logs <pod> --previous — not plain logs, which shows the current (already-crashed-and-restarted) attempt and can be nearly empty — to see what the dead container actually printed before it died. One cause worth checking explicitly and separately: a misconfigured liveness probe can look identical to an application crash from the outside, because kubelet kills the container itself when the probe fails, and if the probe's threshold is tighter than the app's real startup time, a perfectly healthy app gets killed before it's ready — that's what startupProbe exists to fix.

What they're really checking: whether you check the exit code before reading logs. The two most common wrong first moves are re-deploying and hoping, or staring at logs without --previous and seeing nothing useful.

🟡 Q8 · Service isn't routing to Pods

As they'll ask it: "Pods are Running, but the Service in front of them isn't sending them any traffic. What's your process?"

Model answer. I isolate which layer is broken before touching anything. First: kubectl get endpoints <service> (or endpointslices). If it's empty, the Service's selector doesn't match the Pods' labels — a typo or a stale selector after a label change, and it's the single most common cause of this exact symptom. If Endpoints has fewer entries than expected, the missing Pods are Running but not Ready — only Ready Pods populate a Service's Endpoints, so a failing readinessProbe silently pulls a Pod out of rotation without it ever showing as unhealthy in get pods.

If Endpoints looks correct, next I check the port mapping — the Service's targetPort has to match what the container is actually listening on, not what I assume it's listening on. Then I test from inside the cluster with a throwaway debug Pod (kubectl run or kubectl debug) doing a direct curl to a Pod IP first, then to the Service's ClusterIP, to separate "the app itself isn't answering" from "the Service isn't forwarding." If the app answers directly but not through the Service, I'd check kube-proxy's rules on the node and whether a NetworkPolicy is blocking the path — see Networking & the CNI for how that layer actually forwards packets.

What they're really checking: whether you have an ordered process — Endpoints, then readiness, then ports, then policy — versus randomly restarting things until it works.

🟡 Q9 · A node goes NotReady

As they'll ask it: "A node shows NotReady in kubectl get nodes. What's happening, and what do you do?"

Model answer. NotReady means kubelet stopped heartbeating to the apiserver within the node-monitor grace period (default around 40 seconds). It doesn't mean the node's workloads are immediately gone — Kubernetes deliberately waits: after the grace period the node gets tainted node.kubernetes.io/unreachable or /not-ready, and Pods without a matching toleration only get evicted and rescheduled after the pod-eviction-timeout (default five minutes), specifically so a brief network blip doesn't trigger a mass reschedule storm.

Causes, roughly by likelihood: the kubelet process itself crashed or is wedged; the node can't reach the apiserver (network partition, an expired kubelet client certificate — this one is sneaky and calendar-driven); the node is under real resource pressure (check kubectl describe node's Conditions block for MemoryPressure/DiskPressure/PIDPressure, each with its own reason and message); or the node genuinely died — a failed VM, a hardware fault. My process: describe node for the Conditions and recent Events first; if I have access, SSH in and check systemctl status kubelet and journalctl -u kubelet for the actual error rather than guessing from the outside.

What they're really checking: whether you know eviction is deliberately delayed and tainted rather than instant — jumping straight to "so all its Pods get rescheduled immediately" is the tell that you haven't actually watched this happen.

🔴 Q10 · describe, logs, or events — and a general method

As they'll ask it: "When do you reach for kubectl describe versus logs versus get events? And more generally — how do you approach a cluster you've never seen before that's misbehaving?"

Model answer. Three different questions, three different tools. describe gives structured facts about one object — its spec, its status conditions, and its own recent Events — the right first move when I already know which object is suspect and want to know why it's in the state it's in. logs (with -c for multi-container Pods and --previous for a crashed one) gives the actual stdout/stderr the process itself wrote — the right tool for "why did the code inside behave this way," which describe can't answer. get events --sort-by=.lastTimestamp is cluster-wide and chronological, useful when I don't yet know which object is the problem, or when the issue isn't attached to any single object's own event list — though events are ephemeral (a default one-hour TTL), so this window closes.

kubectl get events -A --sort-by=.lastTimestamp | tail -30
kubectl describe pod <pod> -n <namespace>
kubectl logs <pod> -n <namespace> --previous -c <container>
kubectl debug <pod> -n <namespace> -it --image=busybox --target=<container>

On an unfamiliar cluster, I go broad before narrow: get pods -A and get events -A first to find what's actually abnormal rather than guessing which Pod to investigate. Then I narrow by layer — scheduling, then kubelet/runtime, then networking, then application — treating each layer as its own search space instead of jumping straight to the application logs. If the built-in tools run out, an ephemeral debug container (kubectl debug) gets me a shell alongside the workload without rebuilding an image just to add curl. A troubleshooting methodology covers this layer-by-layer approach in full.

What they're really checking: whether your knowledge is command-shaped or process-shaped. Anyone can list the three commands; fewer can say which layer each one actually tells you about.

🔴 Q11 · HPA vs VPA vs Cluster Autoscaler

As they'll ask it: "What's the difference between HPA, VPA, and Cluster Autoscaler, and would you ever run HPA and VPA together?"

Model answer. They scale three different dimensions. HPA scales the replica count of a Deployment or StatefulSet, checking observed metrics (CPU/memory via metrics-server, or custom/external metrics via something like Prometheus Adapter) against a target utilisation on a sync loop, with a stabilisation window so it doesn't flap on noise. VPA adjusts requests and limits per Pod based on observed usage history — it's a right-sizing tool, not a traffic-spike tool, because in its default modes applying a new recommendation means evicting and recreating the Pod, which is far too slow to react to a spike. Cluster Autoscaler scales node count in the underlying cloud node pool, triggered by Pods that are Pending due to insufficient cluster capacity (scale-up) or by sustained node underutilisation (scale-down, which respects PodDisruptionBudgets).

Running HPA and VPA together on the same metric is a known way to get oscillation: VPA changes a Pod's CPU request, which changes what "100% of request" even means, which can fight HPA's target-utilisation math and produce a feedback loop. The workable pattern is to split them by dimension — VPA in recommendation-only mode informing manual or periodic request tuning, HPA scaling on a different signal (queue depth, requests-per-second) rather than the same CPU metric VPA is adjusting the denominator of.

What they're really checking: whether you'll reach for VPA to handle a traffic spike, which is the most common misapplication — it's a sizing tool operating on a timescale of hours to days, not seconds.

🟡 Q12 · Setting requests and limits

As they'll ask it: "How would you decide CPU and memory requests and limits for a workload you've never sized before?"

Model answer. Not by eyeballing, and not by picking a round number that "feels safe." I'd start from actual observed utilisation — kubectl top over a representative load period, or better, a VPA recommender running in recommendation-only mode against real traffic — and set the CPU/memory request near typical sustained usage, because requests are what the scheduler bin-packs against; over-requesting wastes capacity fleet-wide, under-requesting causes the scheduler to over-pack nodes that then get contended.

Requests versus limits also decide the Pod's QoS class, which matters under eviction pressure: Guaranteed (requests equal limits on both CPU and memory) gets the strongest eviction protection, Burstable (requests below limits) is the flexible default, and BestEffort (neither set) is evicted first. For the limits themselves I treat CPU and memory differently: memory always gets a real limit, because there's no safe way to let a container "borrow" memory — exceeding it means an OOMKill. CPU I'd either set generously or leave unset, because a tight CPU limit causes throttling even when the node has spare cycles sitting idle, which shows up as latency, not an obvious failure.

What they're really checking: whether you understand requests drive scheduling and limits drive runtime enforcement — they're not the same knob with two settings, they answer two different questions.

🟢 Q13 · OOMKilled vs CPU throttling

As they'll ask it: "What actually happens when a container exceeds its memory limit versus its CPU limit?"

Model answer. Memory is incompressible, CPU is compressible, and that difference is the whole answer. Exceed the memory limit and the kernel's cgroup OOM killer sends the process a hard SIGKILL — the container shows State: Terminated, Reason: OOMKilled, exit code 137 (128 + signal 9), and restarts per the Pod's restart policy; repeated fast OOMKills produce CrashLoopBackOff. There's no graceful degradation available — memory can't be "throttled," only refused.

Exceed the CPU limit and the cgroup CFS quota just throttles the process — it gets fewer scheduling slices, so it runs slower, but nothing crashes and nothing restarts. That makes it a much sneakier problem: a latency-sensitive service can be badly throttled in bursts while its average CPU usage graph looks completely fine, because averaging over a wider window hides short throttled periods. The metric that actually shows it is container_cpu_cfs_throttled_periods_total, not top-line CPU percentage.

What they're really checking: whether you know CPU throttling is silent and memory limits are fatal — conflating the two leads to debugging a "mysterious slowdown" for hours when it's a documented, graphable throttling metric.

🟡 Q14 · PodDisruptionBudgets during scaling

As they'll ask it: "What's a PodDisruptionBudget for, and how does it interact with node draining or Cluster Autoscaler scaling down?"

Model answer. A PDB sets a floor — minAvailable or a ceiling, maxUnavailable — for a group of Pods matched by label selector, and it's enforced against voluntary disruptions only: node drains during maintenance or upgrades, Cluster Autoscaler scaling a node down, or anything going through the Eviction API. It does nothing for involuntary disruptions — hardware failure, a kernel panic, an OOMKill, a spot instance reclaimed out from under you — because those don't ask permission in the first place.

When something tries to evict a Pod in a way that would violate the budget — dropping available replicas below minAvailable — the eviction request is rejected (HTTP 429), and the caller (kubectl drain, Cluster Autoscaler) backs off and retries rather than forcing it through. That protects availability during routine maintenance, at the cost that a drain can stall if the budget is genuinely unsatisfiable — the classic misconfiguration is minAvailable: 100% on a Deployment with a single replica, which makes every eviction of that Pod impossible and blocks the drain indefinitely.

What they're really checking: whether you know the voluntary/involuntary distinction — a candidate who thinks a PDB protects against node hardware failure has a real gap that would surprise them in production.

🟡 Q15 · Scaling a StatefulSet vs a Deployment

As they'll ask it: "How is scaling a StatefulSet different from scaling a stateless Deployment?"

Model answer. A Deployment's replicas are interchangeable — any Pod can be created or removed in any order, in parallel, because nothing downstream cares which specific replica it is. A StatefulSet gives up that interchangeability on purpose: each replica gets a stable, ordinal identity (pod-0, pod-1, ...) addressable individually through a headless Service, and each ordinal gets its own PersistentVolumeClaim via volumeClaimTemplates that follows it across reschedules rather than being shared or recreated fresh.

Scaling is ordered by default: pod-0 must be Running and Ready before pod-1 is created, and on scale-down the highest ordinal terminates first. That ordering exists because many stateful systems — databases, quorum-based systems like etcd, ZooKeeper, or Kafka — need deterministic join/leave order for their own clustering or replication protocol to stay consistent; a Deployment's arbitrary parallel churn would break that. The practical costs: scale-out is serial, not parallel, so it's slower; and scaling down does not delete the PVCs by default, because losing data on a routine scale-down would be far worse than wasting some storage — you have to clean up orphaned PVCs yourself, or opt into persistentVolumeClaimRetentionPolicy (Kubernetes 1.27+) if automatic deletion is actually what you want. See stateful workloads & database operators.

What they're really checking: whether you understand the ordering and storage-retention behaviour is deliberate design for consistency, not a limitation — "why doesn't Kubernetes just parallelise it" is the natural follow-up, and the answer is that the workload's own protocol wouldn't survive it.

🟢 Q16 · Role, ClusterRole, RoleBinding, ClusterRoleBinding

As they'll ask it: "Explain RBAC in Kubernetes — what's the difference between all four of those objects?"

Model answer. Role is a namespaced set of permission rules — which verbs (get, list, watch, create, update, patch, delete) are allowed on which resources, optionally scoped down to named resources. ClusterRole is the same shape but cluster-scoped, used two ways: to grant genuinely cluster-wide permissions (like reading Nodes, which aren't namespaced at all), or to define a reusable rule set that gets bound per-namespace via a RoleBinding — a common pattern to avoid copy-pasting an identical Role into every namespace. RoleBinding grants the permissions in a Role or a ClusterRole to subjects — a User, a Group, or a ServiceAccount — scoped to one namespace. ClusterRoleBinding grants a ClusterRole's permissions cluster-wide, across every namespace.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: team-a
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: team-a
  name: read-pods-ci
subjects:
- kind: ServiceAccount
  name: ci-deployer
  namespace: team-a
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

In practice, least privilege means never binding cluster-admin broadly, avoiding wildcard verbs and resources in real rules, giving every workload its own named ServiceAccount instead of the namespace default, and turning off automountServiceAccountToken where a Pod has no business calling the API at all.

What they're really checking: whether the namespaced/cluster-scoped split is actually clear to you — this is the single most common source of an over-permissioned cluster, granted by someone who reached for a ClusterRoleBinding because it "just worked" without checking blast radius.

🟡 Q17 · Admission controllers, mutating vs validating

As they'll ask it: "What's an admission controller? What's the difference between mutating and validating?"

Model answer. Admission controllers run after a request has already passed authentication and authorization, as the last gate before an object is persisted to etcd. Mutating admission runs first and can change the object — injecting a sidecar (an Istio proxy, a Vault Agent), setting defaults, rewriting an image tag to a pinned digest. Validating admission runs second, against the (possibly now-mutated) object, and can only accept or reject — it can't change anything. That ordering is fixed and matters: whatever validation sees is post-mutation, not the original request.

In practice, policy engines like OPA Gatekeeper or Kyverno implement the validating side as webhooks enforcing rules like "no privileged containers," "every container must declare resource requests," or "images must come from an approved registry." The built-in Pod Security Admission (replacing the deprecated PodSecurityPolicy since 1.25) is a validating check enforced per namespace via a label (pod-security.kubernetes.io/enforce: restricted), and there are also compiled-in controllers unrelated to webhooks at all — ResourceQuota, LimitRanger, NamespaceLifecycle — running in the same chain.

What they're really checking: whether you know mutation happens before validation, not after — get that backwards and you'll design a policy that validates the wrong version of the object.

🟡 Q18 · securityContext and Pod Security Admission

As they'll ask it: "How do you restrict what a container is allowed to do at runtime?"

Model answer. At the Pod or container level, securityContext is the actual lever: runAsNonRoot/runAsUser so the process isn't root inside the container; readOnlyRootFilesystem so a compromised process can't write to its own image layer; allowPrivilegeEscalation: false; dropping all Linux capabilities and adding back only the specific ones actually needed rather than keeping the broad default set; a seccomp profile (RuntimeDefault at minimum); and never setting privileged: true or the host namespaces (hostNetwork, hostPID, hostIPC) unless there's a genuine, narrow reason — each of those punches a hole straight through the container isolation boundary.

Pod Security Admission is how you enforce this at the namespace level instead of trusting every manifest author to remember it — a label like pod-security.kubernetes.io/enforce: restricted makes the apiserver itself reject any Pod that doesn't meet the restricted profile (non-root, capabilities dropped, no host namespaces, seccomp required), with separate audit and warn modes for rolling it out without breaking things first. It's declarative and namespace-scoped, which is a large part of why it replaced PodSecurityPolicy's much more complex binding model.

What they're really checking: whether you reach for enforcement at the namespace level (so it's not optional per-team) rather than only per-manifest convention, which quietly stops being followed the moment someone's in a hurry.

🔴 Q19 · Hardening Secrets

As they'll ask it: "What's wrong with native Kubernetes Secrets by default, and how would you actually harden secrets management?"

Model answer. Native Secrets are base64-encoded, not encrypted — base64 is a reversible encoding, not a cipher — and unless you've explicitly turned on encryption at rest (an EncryptionConfiguration on the apiserver, ideally backed by a cloud KMS provider so the key isn't sitting on the same disk as what it protects), they sit in etcd in effectively plaintext. Anyone with RBAC read access to the Secret object, or shell access to a Pod that mounts it, can read the value directly.

Hardening, roughly in order of leverage: enable etcd encryption at rest with a real KMS-backed provider first, since that's the baseline everything else assumes; treat read access to Secrets as a deliberate RBAC scoping decision rather than an afterthought bundled into a broad Role; prefer short-lived, dynamically issued credentials over long-lived static ones wherever the target system supports it — HashiCorp Vault's Kubernetes auth method with the Vault Agent injector, or a cloud provider's workload identity federation so a Pod's ServiceAccount assumes a cloud IAM role directly, with no static cloud key ever stored as a Secret at all. For GitOps workflows, encrypt secrets in git rather than committing plaintext manifests (Sealed Secrets, or SOPS), and let DevSecOps's secrets management lesson cover the rotation mechanics in full — this course focuses on the Kubernetes-specific surface, not the general discipline.

What they're really checking: whether "base64" and "encrypted" are the same thing to you. This is one of the most reliable gaps between candidates who've only used Secrets and candidates who've actually had to defend the choice to a security review.

Common wrong answer: "Kubernetes Secrets are already encrypted, so we're fine." That's the exact gap this question is designed to surface — encoded and encrypted are not the same word.

🟡 Q20 · NetworkPolicy default-deny

As they'll ask it: "How do you reason about east-west traffic control between services in a cluster?"

Model answer. By default every Pod can reach every other Pod — a flat network, no isolation — unless the cluster's CNI plugin actually enforces NetworkPolicy at all (not all do; Calico and Cilium enforce it, a bare bridge or plain flannel doesn't). The moment any NetworkPolicy selects a Pod, that Pod becomes default-deny for whichever direction (ingress and/or egress) the policy's policyTypes covers — only traffic matching an explicit allow rule gets through. A Pod with zero policies selecting it stays fully open.

The pattern I'd apply: start with a namespace-wide default-deny baseline — an empty podSelector matching everything, both policy types, no rules — then layer narrow allow rules per service pair by label or namespace selector.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  namespace: checkout
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes: ["Ingress", "Egress"]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  namespace: checkout
  name: allow-from-ingress-controller
spec:
  podSelector:
    matchLabels: { app: checkout-api }
  policyTypes: ["Ingress"]
  ingress:
  - from:
    - namespaceSelector:
        matchLabels: { kubernetes.io/metadata.name: ingress-nginx }
    ports:
    - { protocol: TCP, port: 8080 }

That's the same whitelist-by-default posture as RBAC, just enforced at layer 3/4 instead of the API layer — and it's specifically what limits lateral movement if a single Pod gets compromised. See networking & the CNI for the enforcement mechanics.

What they're really checking: whether you know a policy-free cluster is fully open by default — a candidate who assumes Kubernetes networking is isolated unless you punch holes in it has the model exactly backwards.

🟡 Q21 · "Tell me about an incident you handled in Kubernetes."

As they'll ask it: "Walk me through a real Kubernetes incident you were part of, start to finish."

Model answer. Use Situation/Task/Action/Result, but the part that makes this answer specifically credible is naming the actual Kubernetes-level signal, not a vague "the app went down." Something like: the Deployment I'd shipped ten minutes earlier started showing every replica flapping between Ready and NotReady; kubectl get pods showed 1/1 Running but the Service's Endpoints list was nearly empty. That's a specific, checkable symptom, and it signals I actually watched it happen rather than reconstructing a plausible-sounding story afterward.

Action should be the commands in roughly the order I ran them — describe pod for the readiness probe's failure reason, confirming it was pointed at an endpoint that didn't exist yet in the new image, then kubectl rollout undo to get back to the last-known-good ReplicaSet rather than trying to hot-fix forward under pressure. Result: time to mitigate, and the systemic follow-up — a smoke test added to the pipeline that specifically hits the readiness endpoint before promoting a rollout, not just "I'll double-check probes next time."

What they're really checking: whether the story is real. Vague symptoms and no commands is the tell of a rehearsed-but-hollow answer; specific kubectl output is the tell of someone who was actually there.

🔴 Q22 · "Tell me about a time you caused an outage."

As they'll ask it: "Have you ever shipped something that broke production? What happened?"

Model answer. Own it plainly and specifically — the interviewer is watching how you talk about your own mistake more than the mistake itself. A concrete shape: "I shipped a Deployment where the readiness probe pointed at /health, but that route didn't exist until a later commit in the same image — every replica of the rollout failed readiness simultaneously, the old ReplicaSet had already scaled down under the default rolling-update strategy, and for about four minutes the Service had zero healthy backends." Then the actual response, in order: rolled back immediately rather than trying to patch forward live, confirmed traffic recovered, then investigated why it happened rather than treating the rollback as the end of the story.

The reflection is what separates this from a confession — the fix has to be systemic, not a promise to be more careful. In this case: a pre-promotion smoke test that actually hits the readiness endpoint, and a maxUnavailable: 0 rollout strategy for that Service specifically, so a bad rollout can't take the old, working replicas down before the new ones prove themselves. "I'll be more careful reading my own YAML" is the answer that signals you haven't actually figured out why it happened.

What they're really checking: whether you'll be safe to have in the room after a real outage — someone who reaches for a system fix rather than a personal promise, and who tells the story without quietly routing blame elsewhere.

🟡 Q23 · Debugging with limited observability

As they'll ask it: "Tell me about a time you had to debug a Kubernetes problem with basically no tooling — no dashboards, no established runbook."

Model answer. The honest version of this story usually involves falling back to what Kubernetes itself already tracks before reaching for anything fancier — describe's Events and Conditions, get events cluster-wide, an ephemeral debug container to get a shell alongside a Pod that had no debugging tools baked into its image, and if it came to it, tcpdump or a plain curl loop from inside the cluster to see what a request path actually did rather than trusting an assumption about it. The point worth making explicitly is that this is slower and higher-risk than working with real observability, and I'd say so rather than pretend the manual approach is just as good.

The result that matters most in this story isn't just that the immediate problem got solved — it's what got built afterward so the next person isn't blind in the same way: structured logs with a correlation ID, a basic dashboard on the four golden signals, an alert on the specific symptom that took so long to notice manually. See observability on Kubernetes for what that baseline should actually look like.

What they're really checking: resourcefulness under real constraints, and whether you'll advocate afterward for the tooling that would have made it faster — rather than quietly repeating the same slow manual process next time.

🟡 Q24 · Disagreeing on incident response

As they'll ask it: "Tell me about a time you disagreed with a teammate about how to respond to a live incident."

Model answer. A concrete, common version: mid-incident, a teammate wants to immediately delete and recreate a resource that's stuck in a bad state — a PVC that won't bind, a namespace hanging in Terminating — because it's fast and usually works. I'd push back on the immediate delete specifically for a stuck PVC, because deleting it without first checking the PersistentVolume's reclaim policy can permanently lose data if it's set to Delete rather than Retain; for a stuck Terminating namespace, force-removing finalizers can leave orphaned cloud resources (load balancers, disks) that nothing in the cluster tracks anymore.

The action that matters is how I disagreed — stating the specific risk rather than a vague "I don't think we should," proposing the ten-second check that would tell us whether the fast path was actually safe here, and being genuinely willing to be wrong if the check came back clean. Result: whichever way it actually went, including a version where I was overruled and it turned out fine, or where the check caught something real. A story where you were obviously right and they were obviously careless reads as unreflective — the honest, less flattering version is usually the stronger answer.

What they're really checking: whether you can disagree without it turning personal under real time pressure, and whether you're someone who'll actually check instead of just asserting confidence louder.

🔴 Q25 · Running a blameless postmortem

As they'll ask it: "How do you run a postmortem after a Kubernetes incident, and what makes it blameless in practice rather than just in name?"

Model answer. Blameless means separating "what broke and why" from "whose fault is it," and only asking the first question — not because nobody's accountable, but because punishing the second question is what makes people stop admitting near-misses, which is exactly the information the review needs. In practice for a Kubernetes incident that means reconstructing an actual timeline from evidence, not memory: kubectl get events if it's still within the TTL window, cluster audit logs for who changed what and when, the CI/CD deploy log correlated against the incident window, and the git history of whatever manifest or Helm value changed.

The review treats a bad readiness probe, a missing PodDisruptionBudget, or an unreviewed RBAC change as a system gap — missing validation, missing canary step, missing review requirement — not a personal failing of whoever wrote it, because "an engineer did the reasonable thing and it broke anyway" is a design defect, not a training defect. It produces owned, dated follow-up actions, not a feeling of closure — and I'd track whether those actions actually landed, because a postmortem whose action items quietly evaporate teaches the org that postmortems don't matter. See SRE's postmortems & blameless culture and incident management & on-call for the general discipline this specialises.

What they're really checking: whether "blameless" is a word you've learned or a practice you've actually run — the tell is whether your answer includes concrete evidence sources and dated action items, or stays at the level of "we don't point fingers."