Capstone Part 5 — Security & RBAC

This is the fifth and last of five parts building one continuous cluster: orbit-api, the same satellite-tracking service you deployed as a real Deployment in Part 2, gave a stable network address in Part 3, and handed a persistent store in Part 4. Every part before this one has been running under Kubernetes' loosest defaults on purpose — the default ServiceAccount, no Pod Security enforcement, no NetworkPolicy at all — because proving each layer worked came first, and locking it down before it worked would have just meant debugging two problems at once. Today you stop deferring that debt. You give orbit-api its own identity instead of the default one, scope a CI pipeline's RBAC down to exactly the one Deployment it's allowed to touch, narrow who can actually read the Secret Part 2 flagged as too widely readable, enforce Kubernetes' restricted Pod Security Standard and fix the Deployment so it actually passes, and gate every packet reaching orbit-api behind a default-deny NetworkPolicy — which you'll prove is real by breaking it yourself with a one-character typo, diagnosing it the way a live incident would force you to, and fixing it. By the end of this page there is no more "later" left in this capstone.

What's arriving from Part 4, and what "locked down" means on this page

☺ Like you're 10: Four chapters of a running building, no locks yet — today you fit exactly three, and each one checks something completely different.

This part assumes Part 4 left you with the same capstone namespace every part since Part 2 has been building in, orbit-api still healthy inside it, and its persistent store attached. Confirm both before touching anything below — these two checks don't depend on exactly how Part 3 or Part 4 wired their own pieces, only that they're there and healthy:

kubectl get ns capstone                          # expect: Active
kubectl get deployment orbit-api -n capstone      # expect: 2/2 READY

If either comes back empty or unhealthy, go finish Capstone Part 4 — Storage & Stateful Apps first. Five parts share one running project, so it's worth closing out the shape one final time, here, before changing anything:

"Locked down" is doing three genuinely different jobs on this page, not one blanket tightening — and keeping them mentally separate is the whole point, the same distinction RBAC & Admission Control draws in depth. RBAC answers who is allowed to ask the Kubernetes API for something. Pod Security Admission answers what shape a Pod is allowed to have before it's ever created — privileges, capabilities, the user it runs as. NetworkPolicy answers where a packet is allowed to travel once a Pod is already running. Security: Defense in Depth maps five layers across a cluster's whole lifecycle; this page builds three of them — the two write-time layers and the continuous network layer — deliberately leaving the before-the-cluster image-scanning layer and the runtime-detection layer (Falco and friends) to that deep dive rather than re-deriving them here.

Giving orbit-api its own identity — and a token it never needed

☺ Like you're 10: Every worker in the building has been wearing the same visitor badge since day one. Nobody's abused it yet. That's not the same as it being the right badge.

orbit-api has been running under the capstone namespace's default ServiceAccount since Part 2 — the identity every Pod gets automatically when nothing else is specified. orbit-api never once calls the Kubernetes API: it's a plain Express service that reads its own env vars and answers HTTP requests. A Pod that never talks to the API still gets a bound, projected ServiceAccount token mounted into its filesystem by default, exactly the mechanism RBAC & Admission Control covers — and an unused credential sitting in a container's filesystem is still a credential a compromised container could try to use. Give it a dedicated identity with nothing granted to it, and turn the token off entirely:

# serviceaccount-orbit-api.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: orbit-api
  namespace: capstone
automountServiceAccountToken: false   # orbit-api never calls the Kubernetes API — it doesn't need one

# deployment.yaml — one line added to the Pod template Part 2 wrote
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orbit-api
  namespace: capstone
  labels: { app: orbit-api, part-of: capstone }
spec:
  replicas: 2
  selector:
    matchLabels: { app: orbit-api }
  template:
    metadata:
      labels: { app: orbit-api, part-of: capstone }
    spec:
      serviceAccountName: orbit-api   # new — was implicitly "default" before this line existed
      containers:
        - name: orbit-api
          image: orbit-api:v1
          # ...everything else here is exactly what Part 2 already wrote

kubectl apply -f serviceaccount-orbit-api.yaml
kubectl apply -f deployment.yaml
kubectl rollout status deployment/orbit-api -n capstone

POD=$(kubectl get pod -n capstone -l app=orbit-api -o jsonpath='{.items[0].metadata.name}')
kubectl exec -n capstone "$POD" -- ls /var/run/secrets/kubernetes.io/serviceaccount
# ls: /var/run/secrets/kubernetes.io/serviceaccount: No such file or directory

That error is the proof, not a formality — before this change, that same path held a real, live, auto-rotated token, even though nothing in orbit-api ever read it. Now there's nothing there to leak.

Namespaced RBAC: a scoped ServiceAccount for CI, and only naming who reads the Secret

☺ Like you're 10: The delivery contractor's badge should open the loading dock. It should not also open the safe, just because nobody ever bothered to cut a narrower key.

A real pipeline needs to roll out new orbit-api images without a human running kubectl apply by hand every time. Give it exactly that — patch and read orbit-api's own Deployment, read Pod status and logs to confirm a rollout — and nothing else. Not other namespaces, not Secrets, not the ability to create new RBAC objects for itself:

# rbac-ci-deployer.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: orbit-ci-deployer
  namespace: capstone
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: orbit-ci-deployer
  namespace: capstone
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    resourceNames: ["orbit-api"]
    verbs: ["get", "patch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["list", "watch"]      # resourceNames can't scope list/watch — see the callout below
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: orbit-ci-deployer
  namespace: capstone
subjects:
  - kind: ServiceAccount
    name: orbit-ci-deployer
    namespace: capstone
roleRef:
  kind: Role
  name: orbit-ci-deployer
  apiGroup: rbac.authorization.k8s.io

Apply it and prove the ceiling with kubectl auth can-i's own --as impersonation — no real token needed to check what an identity can do:

kubectl apply -f rbac-ci-deployer.yaml

kubectl auth can-i patch deployment/orbit-api -n capstone \
  --as=system:serviceaccount:capstone:orbit-ci-deployer
# yes

kubectl auth can-i delete deployment/orbit-api -n capstone \
  --as=system:serviceaccount:capstone:orbit-ci-deployer
# no — delete was never granted, only get and patch

kubectl auth can-i get secret/orbit-api-secrets -n capstone \
  --as=system:serviceaccount:capstone:orbit-ci-deployer
# no

kubectl auth can-i list namespaces \
  --as=system:serviceaccount:capstone:orbit-ci-deployer
# no — a Role only ever grants inside its own namespace, never cluster-wide

Now close the gap Part 2 named outright and deferred to this page: "anyone with get on this Secret object... can decode API_TOKEN in one base64 -d." Name exactly who that should be — a Role scoped by resourceNames to this one Secret, bound to a group standing in for the humans actually on call for orbit-api, not to every identity in the cluster by omission:

# rbac-secrets-reader.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: orbit-secrets-reader
  namespace: capstone
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["orbit-api-secrets"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: orbit-secrets-reader-oncall
  namespace: capstone
subjects:
  - kind: Group
    name: "capstone:oncall"          # stands in for whatever real identity your IdP hands out
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: orbit-secrets-reader
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f rbac-secrets-reader.yaml

kubectl auth can-i get secret/orbit-api-secrets -n capstone \
  --as=nobody --as-group=capstone:oncall
# yes — the named group, and only the named group

kubectl auth can-i get secret/orbit-api-secrets -n capstone \
  --as=system:serviceaccount:capstone:orbit-ci-deployer
# still no — the CI identity was never in that group

kubectl auth can-i get secret/orbit-api-secrets -n capstone \
  --as=system:serviceaccount:capstone:orbit-api
# still no — orbit-api's own identity doesn't need to read its own Secret; it's handed the value via secretKeyRef at Pod creation, not by calling the API itself

Nothing here needed a real OIDC provider or an actual second human account to prove — --as and --as-group impersonation is exactly how you audit a binding before anyone real is depending on it, on a lab cluster or a production one.

Pod Security Admission at restricted: label the namespace, then actually fix the Pod spec

☺ Like you're 10: The dress-code inspector doesn't care that you've worked here four chapters without incident — the coat still needs its zip up, today, before you walk through.

RBAC & Admission Control covers Pod Security Admission as a built-in plugin enforcing the Pod Security Standards per namespace, at three levels: privileged, baseline, restricted. capstone currently carries no pod-security.kubernetes.io/* labels at all, which means it defaults to privileged — the loosest level, enforcing nothing. Turn on the strictest level in all three modes at once, so nothing slips through silently:

kubectl label ns capstone \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=latest \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/audit=restricted \
  --overwrite

The label change alone doesn't touch orbit-api's two Pods already running — PSA only ever evaluates a Pod at the moment it's created, never retroactively. The next thing that tries to create a new one is where the truth comes out:

kubectl rollout restart deployment/orbit-api -n capstone
# deployment.apps/orbit-api restarted   ← this command succeeds; it only patches the Deployment

kubectl rollout status deployment/orbit-api -n capstone --timeout=30s
# Waiting for deployment "orbit-api" rollout to finish... (times out)

kubectl get events -n capstone --field-selector reason=FailedCreate --sort-by=.lastTimestamp | tail -n1
# Warning  FailedCreate  replicaset/orbit-api-7f9c8d6b5b  Error creating: pods "orbit-api-7f9c8d6b5b-" is
# forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container
# "orbit-api" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities
# (container "orbit-api" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true
# (pod or container "orbit-api" must set securityContext.runAsNonRoot=true), seccompProfile (pod or
# container "orbit-api" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

The rollout itself never errors — it's the new ReplicaSet's attempt to create a Pod that PSA rejects, which is exactly why kubectl rollout status hangs rather than failing outright: the old, still-running, still-privileged-era Pods are never touched, so the Deployment just sits waiting on replacements that can never come into existence. The message names every missing field precisely — nothing to guess at. Fix the Pod template so it actually satisfies restricted, using the node user Node's own official image already ships with:

# deployment.yaml — securityContext added to the Pod template
    spec:
      serviceAccountName: orbit-api
      securityContext:                        # pod-level — applies to every container unless overridden
        runAsNonRoot: true
        runAsUser: 1000                        # the "node" user node:22-alpine already ships with
        runAsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: orbit-api
          image: orbit-api:v1
          securityContext:                      # container-level — capabilities only exists here
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
          # ...ports, envFrom, env, resources, and all three probes unchanged from Part 2

kubectl apply -f deployment.yaml
kubectl rollout status deployment/orbit-api -n capstone
# deployment "orbit-api" successfully rolled out

kubectl get pods -n capstone -l app=orbit-api \
  -o jsonpath='{range .items[*]}{.metadata.name}{"  uid="}{.spec.securityContext.runAsUser}{"\n"}{end}'
# orbit-api-6b8f7d9c4-abcde  uid=1000
# orbit-api-6b8f7d9c4-fghij  uid=1000

Same image, same ConfigMap, same Secret reference, same three probes — the only thing that changed is the shape PSA now requires, and now gets.

Defense in depth's last layer: default-deny, and this capstone's own troubleshooting drill

☺ Like you're 10: RBAC checked the badge. Pod Security checked the coat. Nobody's checked yet whether the front-desk guard is even reading the visitor list correctly — until today, when you hand them a list with one name misspelled and watch what happens.

Part 1 installed a real Calico CNI instead of kind's default kindnet specifically so this section would be possible — kindnet enforces no NetworkPolicy at all, so practicing this on top of it would just be writing YAML nothing ever reads. Start with a blanket deny, in both directions, for the whole namespace:

# netpol-default-deny.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: capstone
spec:
  podSelector: {}                 # every Pod in this namespace
  policyTypes: ["Ingress", "Egress"]

kubectl apply -f netpol-default-deny.yaml

curl -m 5 http://orbit.localtest.me/healthz/ready   # the Ingress host Part 3 wired up — adjust if yours differs
# curl: (28) Connection timed out after 5001 milliseconds

That timeout is the proof the policy is real — a moment ago that same request answered. Now add back exactly the traffic orbit-api actually needs: ingress from ingress-nginx's own namespace on its one port, egress to CoreDNS, and egress to whatever Part 4's storage listens on within this namespace. Every namespace carries an automatic, immutable kubernetes.io/metadata.name label matching its own name since Kubernetes 1.21, which is what makes namespace-scoped selectors like the one below possible without hand-labeling anything first:

# netpol-orbit-api-allow.yaml — SABOTAGED ON PURPOSE, see below
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: orbit-api-allow
  namespace: capstone
spec:
  podSelector:
    matchLabels:
      app: orbit_api                # <- typo: an underscore where every real Pod label uses a hyphen
  policyTypes: ["Ingress", "Egress"]
  ingress:
    - from:
        - namespaceSelector:
            matchLabels: { kubernetes.io/metadata.name: ingress-nginx }
      ports:
        - { protocol: TCP, port: 8080 }
  egress:
    - to:
        - namespaceSelector:
            matchLabels: { kubernetes.io/metadata.name: kube-system }
      ports:
        - { protocol: UDP, port: 53 }
        - { protocol: TCP, port: 53 }
    - to:
        - podSelector: {}           # anything else Part 4's storage runs inside this same namespace
      ports:
        - { protocol: TCP, port: 5432 }

kubectl apply -f netpol-orbit-api-allow.yaml
# networkpolicy.networking.k8s.io/orbit-api-allow created   ← accepted. It's syntactically valid YAML.

curl -m 5 http://orbit.localtest.me/healthz/ready
# curl: (28) Connection timed out after 5001 milliseconds     ← still broken

This is the moment a real CKA scenario puts you in: a policy that applied cleanly, an API server that never complained, and traffic that's still dead. Diagnose it the way the exam actually rewards — read the object back, don't re-derive it from memory:

kubectl describe networkpolicy orbit-api-allow -n capstone
# Name:         orbit-api-allow
# Spec:
#   PodSelector:     app=orbit_api
#   Allowing ingress traffic:
#     ...

kubectl get pods -n capstone --show-labels
# NAME                         ...   LABELS
# orbit-api-6b8f7d9c4-abcde    ...   app=orbit-api,part-of=capstone
# orbit-api-6b8f7d9c4-fghij    ...   app=orbit-api,part-of=capstone

There it is: the policy's own PodSelector reads app=orbit_api, and every real Pod is labeled app=orbit-api. A selector that matches zero Pods isn't an error Kubernetes can catch for you — an empty match is a perfectly valid NetworkPolicy, it's just a policy that governs nothing, while the namespace's default-deny-all from a moment ago keeps doing exactly what it says regardless. Fix the one character and re-apply:

# netpol-orbit-api-allow.yaml — the only change: app: orbit_api  ->  app: orbit-api
kubectl apply -f netpol-orbit-api-allow.yaml

curl -m 5 http://orbit.localtest.me/healthz/ready
# {"status":"ready","region":"ap-south-1"}   ← real traffic again, gated by a policy that now actually matches something

What "done" looks like — the capstone, closed

☺ Like you're 10: Every lock fitted, every lock tested by someone actually trying the door — five chapters, one story, finished.

At the end of this part: orbit-api runs under its own ServiceAccount with no token it never needed; orbit-ci-deployer can roll out a new image and read logs and nothing else, proven by real denials on delete, on the Secret, and on anything cluster-scoped; orbit-api-secrets is readable by exactly one named group instead of by omission; the capstone namespace enforces restricted Pod Security, and orbit-api's own Pods now run non-root, capability-dropped, and seccomp-confined; and a default-deny NetworkPolicy plus one correctly-selectored allow rule gate every packet that reaches it — broken and fixed by your own hand, not taken on faith. Nothing from any earlier part was thrown away to get here:

Part 5 closed the capstone: orbit-api runs under its own least-privilege identity, a CI pipeline can deploy it and nothing more, its one Secret is readable by name instead of by accident, its Pods satisfy the strictest Pod Security Standard Kubernetes ships, and every packet reaching it has to clear a NetworkPolicy you've personally watched fail and then work. There is no Part 6 — five parts, one continuous cluster, done. Step back to Build a Cluster — Start Here to see the whole five-stage arc in one place, or turn what you just proved with your own hands into exam-ready recall with RBAC & Admission Control and Security: Defense in Depth. Get a faster, standalone rep of just the RBAC-and-admission workflow with Drill: Harden an RBAC Configuration, or a general troubleshooting rep with Drill — Fix a Broken Cluster. If this page's territory is the direction you're headed next, it's the entire subject of the CKS blueprint — and the nine other CNCF certifications plus the LFCS that complete the Golden Kubestronaut ladder beyond CKS live on the sibling Golden Astronaut course.

Milestones

☺ Like you're 10: Tick a box only once you've watched the real command output on your own screen — a rejection message you read yourself, not one you assumed would appear.

Work these in order — each depends on the RBAC and namespace state from the one before. Progress saves in this browser.