Drill — Harden an RBAC Configuration

A ServiceAccount called release-bot exists in namespace storefront and has exactly one job: let a CI pipeline update the Deployments and ConfigMaps it pushes several times a day. It was wired up two weeks ago the fast way — one ClusterRoleBinding to the built-in cluster-admin ClusterRole, "just to get the pipeline through the demo," and nobody has been back to narrow it since. That's the whole setup for this drill: a self-contained, single-skill exercise with no capstone dependency, roughly 30 minutes end to end on a disposable local cluster. You'll stand up the over-permissive binding yourself, prove exactly what a leaked token for this identity could reach using kubectl auth can-i, design the least-privilege Role that actually matches the job, apply it in place of the ClusterRoleBinding, and verify the new boundary holds — what still works, and what now correctly refuses.

How this drill works

☺ Like you're 10: One over-permissive binding, one properly scoped fix, and you prove the difference with the exact command a real access review would run.

This is a single-skill drill, not a multi-finding investigation like Drill — Fix a Broken Cluster. There's one problem: a real, working ServiceAccount that does a real, useful job, currently holding vastly more power than that job requires. You'll seed it yourself, prove the gap with commands rather than guesswork, replace it with a Role scoped to exactly what release-bot needs, and confirm with kubectl auth can-i that the dangerous permissions are gone while the actual job still works. RBAC & Admission Control covers the mechanics behind every step here — aggregated ClusterRoles, the escalate/bind/impersonate verbs, how bound ServiceAccount tokens work — if anything below doesn't fully click, that's where to go first, then come back and actually do it.

Set up the over-permissive ServiceAccount

☺ Like you're 10: One small cluster, one small pipeline identity, one binding that grants it the keys to absolutely everything.

Stand up a disposable single-node cluster:

kind create cluster --name rbac-drill
kubectl config use-context kind-rbac-drill
kubectl get nodes      # one node, Ready — that's all this drill needs

Create the namespace release-bot actually deploys into, and a small Deployment standing in for the real workload it pushes updates to:

kubectl create namespace storefront
kubectl create serviceaccount release-bot -n storefront

# storefront-web.yaml — the workload release-bot's pipeline actually needs to touch
apiVersion: apps/v1
kind: Deployment
metadata:
  name: storefront-web
  namespace: storefront
spec:
  replicas: 2
  selector:
    matchLabels: { app: storefront-web }
  template:
    metadata:
      labels: { app: storefront-web }
    spec:
      containers:
      - name: web
        image: nginx:1.27
        ports: [{ containerPort: 80 }]

Also create a second namespace standing in for a different team's data — this is the canary that makes the over-permission concrete instead of theoretical:

kubectl create namespace payments
kubectl create secret generic db-creds -n payments --from-literal=password=hunter2
kubectl apply -f storefront-web.yaml

Now seed the actual problem — the binding as it was really written two weeks ago, under deadline pressure, "temporarily":

# release-bot-cluster-admin.yaml — as given: the fast, wrong way to unblock a demo
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: release-bot-cluster-admin
subjects:
- kind: ServiceAccount
  name: release-bot
  namespace: storefront
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f release-bot-cluster-admin.yaml
kubectl get clusterrolebinding release-bot-cluster-admin

Audit what release-bot can actually do

☺ Like you're 10: You don't have to steal the badge to test what it opens — you can just ask the door "would this badge get me in," for any door in the building.

You never need the ServiceAccount's real token to test what it's allowed to do. kubectl auth can-i supports --as impersonation directly, using the exact identity string Kubernetes assigns every ServiceAccount: system:serviceaccount:<namespace>:<name>. Start with the broad view:

kubectl auth can-i --list --as=system:serviceaccount:storefront:release-bot

Resources                                       Non-Resource URLs   Resource Names   Verbs
*.*                                                                                   [*]
                                                  [*]                                 [*]

Two lines say everything. *.* with verbs [*] means every resource, in every API group, with every verb — get, list, delete, all of it. The second line, [*] under Non-Resource URLs, means it can hit any of the apiserver's own HTTP endpoints too, not just object CRUD. That's what cluster-admin actually is: not "a lot of access," but the literal absence of a boundary anywhere. Confirm the specific, concrete damage a leaked release-bot token could do today:

kubectl auth can-i delete secrets -n payments --as=system:serviceaccount:storefront:release-bot
# yes

kubectl auth can-i delete namespaces --as=system:serviceaccount:storefront:release-bot
# yes

kubectl auth can-i create clusterrolebindings --as=system:serviceaccount:storefront:release-bot
# yes

kubectl auth can-i get secrets --all-namespaces --as=system:serviceaccount:storefront:release-bot
# yes

Every one of those is a "yes" for an identity whose entire job is updating one Deployment and its ConfigMaps in one namespace. It can read and delete payments' database credentials, delete the payments namespace outright, and grant itself — or anything else — more ClusterRoleBindings, because bind is one of the verbs a wildcard already includes. A single leaked CI token is currently equivalent to full cluster compromise, and nothing about the pipeline's actual job required any of that.

Design the least-privilege Role

☺ Like you're 10: Before you cut the new key, write down every single door it actually needs to open — no guessing, no "probably fine," just the real list.

List release-bot's real job before writing a single line of RBAC: it updates the image and config on storefront-web, and it watches the rollout to know whether that update succeeded. Nothing in that job touches Secrets, Nodes, Namespaces, or any resource outside storefront.

No delete anywhere, not even on the resources it manages — this pipeline only ever pushes forward; a rollback is a new update, not a deletion. No secrets access at all, in this namespace or any other. Nothing cluster-scoped. Write it as a Role, which is namespace-bound by construction — the API simply has no field for granting a Role reach outside the namespace it's created in:

# role-release-bot.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: release-bot-deployer
  namespace: storefront
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: ["apps"]
  resources: ["replicasets"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get"]

# rolebinding-release-bot.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: release-bot-deployer
  namespace: storefront
subjects:
- kind: ServiceAccount
  name: release-bot
  namespace: storefront
roleRef:
  kind: Role
  name: release-bot-deployer
  apiGroup: rbac.authorization.k8s.io

Apply the fix — replace the ClusterRoleBinding, don't just add on top of it

☺ Like you're 10: Cutting the new key doesn't do anything by itself if the old master key still works too — you have to actually take the old one back.

Adding the new Role and RoleBinding without removing the old ClusterRoleBinding would change nothing — RBAC is additive, so release-bot would simply hold both grants at once, with cluster-admin still silently winning every check. Delete the binding first, then apply the scoped replacement:

kubectl delete clusterrolebinding release-bot-cluster-admin
kubectl apply -f role-release-bot.yaml
kubectl apply -f rolebinding-release-bot.yaml

Verify with kubectl auth can-i

☺ Like you're 10: Ask the exact same questions you asked before, and confirm the answers actually changed where they were supposed to — and only there.

Start with the same broad view, now scoped to the one namespace that has any grant at all:

kubectl auth can-i --list --as=system:serviceaccount:storefront:release-bot -n storefront

Resources                                       Non-Resource URLs   Resource Names   Verbs
configmaps                                       []                  []               [create get list patch update watch]
deployments.apps                                 []                  []               [create get list patch update watch]
pods                                              []                  []               [get list watch]
pods/log                                          []                  []               [get]
replicasets.apps                                 []                  []               [get list watch]

Five rows instead of one wildcard row — exactly the table you designed above, nothing more. Re-run the specific checks from the audit step and confirm each one flips, then confirm the actual job still works:

kubectl auth can-i delete secrets -n payments --as=system:serviceaccount:storefront:release-bot
# no

kubectl auth can-i delete namespaces --as=system:serviceaccount:storefront:release-bot
# no

kubectl auth can-i create clusterrolebindings --as=system:serviceaccount:storefront:release-bot
# no

kubectl auth can-i update deployments -n storefront --as=system:serviceaccount:storefront:release-bot
# yes — the actual job still works

kubectl auth can-i update deployments -n payments --as=system:serviceaccount:storefront:release-bot
# no — same resource type, wrong namespace, correctly denied

That last check is the one people forget to run. It's not enough that the dangerous, unrelated permissions are gone — the fix also has to prove that the real permission it kept is scoped to the one namespace the job actually runs in, not just narrowed in which verbs it allows.

Boundary holds? That's the whole drill. For the full mechanics behind every step here — aggregated ClusterRoles, bound ServiceAccount tokens, and the admission-control layer that runs independently of RBAC — see RBAC & Admission Control and Security: Defense in Depth; for the CKA-level Role/RoleBinding baseline this drill assumes, see Cluster Architecture, Installation & Configuration. DevSecOps' Kubernetes security deep dive covers Pod Security Standards and policy engines at full depth if you took on the going-further challenge above. Practice the same tightening pass again, at full cluster scale alongside NetworkPolicy and Pod Security, in Capstone Part 5 — Security & RBAC — or start the capstone from the beginning at Build a Cluster — Start Here. Heading toward the certification this exact habit is built for? See the CKS blueprint, and the wider CNCF ladder beyond it on the sibling Golden Astronaut course. Want a different single skill next? Try Drill — Fix a Broken Cluster.