Capstone Part 6: Security, Tenancy & the Golden Path

This is the last of six parts, and it does two jobs at once. First, it installs the guardrails a real platform needs: policy-as-code that starts in audit and graduates to enforce, RBAC scoped to a tenant instead of cluster-admin, a default-deny NetworkPolicy, mutual TLS from a service mesh, a supply-chain scan, and multi-tenant resource fairness with cost visibility. Second — and this is the payoff of the whole capstone — it wires Parts 1 through 5 together so a brand-new service scaffolded in Backstage flows through GitOps, a pipeline, a canary rollout, a platform CRD, dashboards, and every guardrail on this page automatically, with zero hand-written infrastructure. That is the golden path this entire course has been describing.

⚖ CNPA vs CNPE — This page's hands-on build — Kyverno in enforce, scoped tenant RBAC, default-deny NetworkPolicy, Linkerd mTLS, a Trivy scan, and the golden path itself — is CNPE-specific: CNPE is performance-based, so a narrow allowlist (kubernetes.io/docs, kubernetes.io/blog, and any links in a task's own Quick Reference box) stays reachable mid-task. CNPA has no allowlist at all — it's a fully closed-book multiple-choice exam with zero external lookups of any kind, stricter than CNPE, not looser — and no lab component whatsoever, so nothing on this page is something you'd ever build for it. Even so, the concept layer — why audit precedes enforce, what least-privilege RBAC proves, why NetworkPolicy and mTLS operate at different layers — is exactly the kind of concept-level knowledge CNPA's closed-book recall draws on.

What this part assumes and what it produces

☺ Like you're 10: A quick roll call of everything already standing, so nothing below surprises you.

Before touching anything, confirm the state you should already be in. If any of these are missing, that part isn't finished yet — go back before continuing:

This part adds one new namespace to the world: storefront, a second tenant standing in for another team on the same cluster, used only for the multi-tenancy exercise later on this page. Every other name — platform-dev, ledger, platform, platform-capstone, registry.local/ledger — is exactly what Part 1 established and every later part kept using.

Kyverno: policy-as-code, audit before enforce

☺ Like you're 10: First the guard just writes down who's breaking the rule; only once everyone's fixed do they start turning people away.

Install Kyverno into platform the same GitOps way everything else in this capstone arrives — commit its Application into apps/ so the root App-of-Apps picks it up:

# apps/kyverno.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: kyverno
  namespace: platform
spec:
  project: default
  source:
    repoURL: https://github.com/kyverno/kyverno.git
    targetRevision: main
    path: charts/kyverno
    helm:
      releaseName: kyverno
  destination:
    server: https://kubernetes.default.svc
    namespace: platform
  syncPolicy:
    automated: { prune: true, selfHeal: true }
    syncOptions: [ CreateNamespace=true ]

Commit a policy that requires every pod to run as non-root and forbids the :latest tag — two of the most common accidental holes. Start it in Audit, the way a senior engineer rolls out a new guardrail on a live cluster:

# platform-addons/kyverno/require-non-root-and-pinned-tags.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-non-root-and-pinned-tags
spec:
  validationFailureAction: Audit    # step 1: report only, block nothing yet
  background: true                   # also scan pods that are already running
  rules:
    - name: run-as-non-root
      match:
        any:
          - resources: { kinds: ["Pod"] }
      validate:
        message: "Running as root is not allowed — set runAsNonRoot: true."
        pattern:
          spec:
            securityContext:
              runAsNonRoot: true
    - name: disallow-latest-tag
      match:
        any:
          - resources: { kinds: ["Pod"] }
      validate:
        message: "Images must be pinned to a specific tag, not :latest."
        pattern:
          spec:
            containers:
              - image: "!*:latest"

Commit this into platform-addons/kyverno/ and add an Application for it in apps/, exactly like every other add-on so far. Once it's Synced, read the report — it should already show something, because the Part-1 placeholder and a few early manifests were never written with this rule in mind:

kubectl get clusterpolicyreport -o wide
kubectl get policyreport -n ledger -o wide
# look for POLICY require-non-root-and-pinned-tags, RESULT fail — that's audit doing its job

Fix every violation the report names (add securityContext.runAsNonRoot: true to the ledger Deployment or Rollout, pin the pipeline's image tag), push the fix through Git, and confirm the report clears. Only then flip the switch:

# same file, one line changed and pushed
spec:
  validationFailureAction: Enforce   # step 2: now it blocks

Prove enforcement with a pod that deliberately breaks the rule — it must be rejected at admission, not merely reported:

kubectl run rootcheck --image=nginx:latest -n ledger --dry-run=server -o yaml
# Error from server: admission webhook "validate.kyverno.svc-fail" denied the request:
# policy Pod/ledger/rootcheck for resource violation:
# require-non-root-and-pinned-tags: ... running as root is not allowed ...

Scoped RBAC for the tenant, never cluster-admin

☺ Like you're 10: The ledger team gets a key that opens exactly their own room — not the master key to the whole building.

Give the ledger workload its own ServiceAccount — never the default one — and give the humans on the ledger team a namespaced Role that lets them operate their own Deployments, Services and ConfigMaps but touches nothing outside ledger and nothing security-sensitive inside it:

# platform-addons/rbac/ledger-tenant.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ledger
  namespace: ledger
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: ledger-tenant-dev
  namespace: ledger
rules:
  - apiGroups: ["apps"]
    resources: ["deployments", "replicasets"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["argoproj.io"]
    resources: ["rollouts"]
    verbs: ["get", "list", "watch", "patch"]   # patch: promote/abort a canary, nothing more
  - apiGroups: [""]
    resources: ["services", "configmaps", "pods", "pods/log"]
    verbs: ["get", "list", "watch"]
  # deliberately absent: secrets, networkpolicies, rolebindings, and anything cluster-scoped
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ledger-tenant-dev-binding
  namespace: ledger
subjects:
  - kind: Group
    name: ledger-team
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: ledger-tenant-dev
  apiGroup: rbac.authorization.k8s.io

Set automountServiceAccountToken: false on the ledger Deployment/Rollout's pod spec too, since the ledger workload itself never needs to call the Kubernetes API:

# patch merged into the ledger Rollout's pod template from Part 2
spec:
  serviceAccountName: ledger
  automountServiceAccountToken: false

Prove the ceiling exists by impersonating the tenant identity and trying something outside its Role:

kubectl auth can-i patch rollouts -n ledger --as=system:serviceaccount:ledger:ledger
# yes

kubectl auth can-i delete secrets -n ledger --as=system:serviceaccount:ledger:ledger
# no

kubectl auth can-i get pods -n platform --as=system:serviceaccount:ledger:ledger
# no — the Role is namespaced to ledger, it has no reach into platform

Default-deny NetworkPolicy

☺ Like you're 10: Even with ID checks at every door, you brick up every hallway nobody should walk down.

A plain Kubernetes cluster is allow-all: any pod can reach any other. Fix that in ledger with a default-deny baseline, then open only the flows the app actually needs — including DNS, which a naive default-deny silently breaks:

# platform-addons/netpol/ledger-default-deny.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: ledger
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: ledger
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - namespaceSelector: {}
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress-to-ledger
  namespace: ledger
spec:
  podSelector:
    matchLabels: { app: ledger }
  policyTypes: [Ingress]
  ingress:
    - from:
        - namespaceSelector:
            matchLabels: { kubernetes.io/metadata.name: platform }
      ports:
        - protocol: TCP
          port: 9898

Prove the blast radius actually shrank — spin up a scratch pod in a third namespace and confirm it cannot reach ledger, then confirm the platform namespace still can:

kubectl create namespace scratch
kubectl run probe -n scratch --rm -it --image=busybox:1.36 --restart=Never -- \
  wget -qO- --timeout=3 http://ledger.ledger.svc.cluster.local
# times out — default-deny is doing its job

kubectl run probe -n platform --rm -it --image=busybox:1.36 --restart=Never -- \
  wget -qO- --timeout=3 http://ledger.ledger.svc.cluster.local
# succeeds — this namespace is explicitly allowed

Linkerd mTLS across the mesh

☺ Like you're 10: Every service now shows an ID card and whispers through a private tube nobody else can listen to.

Mesh the ledger and platform namespaces with Linkerd — deliberately lighter than Istio, and it turns mTLS on automatically with zero policy YAML for the encryption itself. Install the CLI and the control plane, running linkerd check before and after as the project itself insists on:

curl -sL https://run.linkerd.io/install | sh
linkerd check --pre

linkerd install --crds | kubectl apply -f -
linkerd install | kubectl apply -f -
linkerd check

linkerd viz install | kubectl apply -f -
linkerd check

Mesh the namespace by annotating it, then restart the workload so it picks up the sidecar:

kubectl annotate namespace ledger linkerd.io/inject=enabled
kubectl -n ledger rollout restart deploy/ledger 2>/dev/null || kubectl -n ledger rollout restart rollout/ledger

kubectl -n ledger get pods
# each ledger pod now shows 2/2 containers — the app, plus linkerd-proxy

Confirm mTLS is live and see the identity on every edge — no certificates to generate by hand, Linkerd derives them from each pod's ServiceAccount:

linkerd viz stat deploy -n ledger
linkerd viz edges deploy -n ledger
# SRC          DST      SRC_IDENTITY                         DST_IDENTITY
# platform-*   ledger   platform.serviceaccount.identity...   ledger.serviceaccount.identity...

Tighten it with an explicit authorization so only the platform namespace's callers (your Prometheus, your ingress) may reach ledger at all — the network-layer complement to the RBAC and NetworkPolicy you already added:

# platform-addons/linkerd/ledger-server-authz.yaml
apiVersion: policy.linkerd.io/v1beta3
kind: Server
metadata:
  name: ledger-9898
  namespace: ledger
spec:
  podSelector:
    matchLabels: { app: ledger }
  port: 9898
  proxyProtocol: HTTP/1
---
apiVersion: policy.linkerd.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ledger-allow-platform
  namespace: ledger
spec:
  targetRef:
    group: policy.linkerd.io
    kind: Server
    name: ledger-9898
  requiredAuthenticationRefs:
    - group: policy.linkerd.io
      kind: MeshTLSAuthentication
      name: platform-callers
---
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata:
  name: platform-callers
  namespace: ledger
spec:
  identities:
    - "*.platform.serviceaccount.identity.linkerd.cluster.local"

Supply-chain: scan the ledger image with Trivy

☺ Like you're 10: Before you eat the sandwich, check the ingredient list for anything that could make you sick.

Scan the exact image your Part-2 pipeline built and pushed to registry.local/ledger. Run it as its own step so a critical finding is visible before anything ships, and again against the live cluster so you know what's actually running:

trivy image --severity CRITICAL,HIGH --ignore-unfixed registry.local/ledger:TAG

trivy image --format json --output ledger-scan.json registry.local/ledger:TAG

# what's actually deployed right now, cluster-wide
trivy k8s --report summary --include-namespaces ledger,platform

Read the findings the way a platform engineer would: a CRITICAL with a fix available is a blocker — bump the base image or the dependency and re-push through the pipeline; a HIGH with no fix yet gets triaged and tracked, not ignored silently. If this were wired into CI (Part 2's pipeline is the natural home for it), a non-zero --exit-code on CRITICAL findings would fail the build before the image is ever pushed — the same "shift left, verify at admission" pattern Security & Policy Enforcement teaches for signing.

Multi-tenancy: a second namespace, fair shares, no starving

☺ Like you're 10: Two school groups share the same playground — a fence keeps them apart, and each gets its own fair pile of swings so one group can't hog them all.

Create storefront as a second tenant namespace standing in for another team on the same cluster, and give both ledger and storefront a ResourceQuota and a LimitRange so neither can starve the other or the cluster:

kubectl create namespace storefront

# platform-addons/tenancy/ledger-quota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: ledger-quota
  namespace: ledger
spec:
  hard:
    requests.cpu: "2"
    requests.memory: 2Gi
    limits.cpu: "4"
    limits.memory: 4Gi
    pods: "20"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: ledger-limits
  namespace: ledger
spec:
  limits:
    - type: Container
      default: { cpu: 250m, memory: 256Mi }         # applied if a container sets no limit
      defaultRequest: { cpu: 100m, memory: 128Mi }  # applied if a container sets no request
      max: { cpu: "1", memory: 1Gi }
      min: { cpu: 50m, memory: 64Mi }
---
# platform-addons/tenancy/storefront-quota.yaml — same shape, its own namespace, its own numbers
apiVersion: v1
kind: ResourceQuota
metadata:
  name: storefront-quota
  namespace: storefront
spec:
  hard:
    requests.cpu: "1"
    requests.memory: 1Gi
    limits.cpu: "2"
    limits.memory: 2Gi
    pods: "10"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: storefront-limits
  namespace: storefront
spec:
  limits:
    - type: Container
      default: { cpu: 250m, memory: 256Mi }
      defaultRequest: { cpu: 100m, memory: 128Mi }
      max: { cpu: "1", memory: 1Gi }
      min: { cpu: 50m, memory: 64Mi }

Isolate them at the network layer too — storefront gets its own default-deny, same shape as ledger's from earlier on this page:

# platform-addons/netpol/storefront-default-deny.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: storefront
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]

Prove the quota is real by trying to exceed it on purpose:

kubectl -n storefront create deployment hog --image=nginx:1.27 --replicas=20 \
  --dry-run=client -o yaml | kubectl apply -f -
kubectl -n storefront get events --sort-by=.lastTimestamp | tail -5
# Warning  FailedCreate  replicaset-controller  Error creating: pods "hog-..." is forbidden:
# exceeded quota: storefront-quota, requested: pods=1, used: pods=10, limited: pods=10

Cost visibility with OpenCost

☺ Like you're 10: Now that everyone shares the same swings fairly, you also want a receipt showing what each group's turn actually cost.

Install OpenCost into its own namespace, GitOps-managed like everything else:

# apps/opencost.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: opencost
  namespace: platform
spec:
  project: default
  source:
    repoURL: https://github.com/opencost/opencost-helm-chart.git
    targetRevision: main
    path: charts/opencost
    helm:
      releaseName: opencost
  destination:
    server: https://kubernetes.default.svc
    namespace: opencost
  syncPolicy:
    automated: { prune: true, selfHeal: true }
    syncOptions: [ CreateNamespace=true ]

Point kubectl cost at the plain-OpenCost install (it defaults to Kubecost's namespace and port, so all four flags below matter) and read spend split by namespace — ledger vs storefront, side by side:

kubectl -n opencost port-forward svc/opencost 9003 9090 &

OC="--kubecost-namespace opencost --service-name opencost --service-port 9003"
kubectl cost namespace $OC --window 24h --show-efficiency
# NAMESPACE     CPU CORE-HRS   RAM GiB-HRS   TOTAL COST   EFFICIENCY
# ledger        ...            ...           $x.xx        ...
# storefront    ...            ...           $y.yy        ...
# platform      ...            ...           $z.zz        ...

That output is the tenancy story made visible in dollars: two isolated, fairly-quotaed namespaces, and now a number showing exactly what each one is spending — the same allocation model OpenCost teaches, now pointed at the tenants you just built.

The finale: wiring Parts 1–5 into one golden path

☺ Like you're 10: Press the "new ride" button at the kiosk once, and watch the entire park — track, fence, wristband and watchtower — build itself around it while you just watch.

Everything above stands alone, but the point of a capstone is that it doesn't stay standing alone. Here is the chain, named once, end to end — this is the sentence to be able to say out loud in the exam or an interview:

Making the template carry the guardrails

The trick that closes the loop is: the guardrails on this page must live inside the Backstage Software Template from Part 4, not be something a human remembers to add afterwards. Extend that template's scaffolded output so every new service's generated ledger/-shaped folder already includes its own NetworkPolicy (default-deny + the narrow allow), its own ResourceQuota/LimitRange, its own scoped ServiceAccount and Role, the linkerd.io/inject: enabled namespace annotation, and a ServiceMonitor pointed at itself — plus an apps/<service>.yaml Application so the root App-of-Apps picks it up without you ever running kubectl apply:

# skeleton/apps/${{values.name}}.yaml — part of the Backstage Software Template's output,
# rendered once per scaffolded service and committed straight into platform-capstone/apps/
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ${{values.name}}
  namespace: platform
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/YOU/platform-capstone.git
    targetRevision: main
    path: ${{values.name}}
  destination:
    server: https://kubernetes.default.svc
    namespace: ${{values.name}}
  syncPolicy:
    automated: { prune: true, selfHeal: true }
    syncOptions: [ CreateNamespace=true ]

Running the golden path live

Now be Dot. Open Backstage, pick the Software Template from Part 4, and scaffold a brand-new service — call it ledger-reports — through the portal, touching no kubectl, no Git client, no YAML editor:

# after clicking "Create" in Backstage and merging the PR it opened, watch the chain fire:
kubectl -n platform get applications -w
# ledger-reports   Synced   Healthy      <- appeared with no kubectl apply from you

kubectl -n ledger-reports get networkpolicy,resourcequota,limitrange,serviceaccount,rollout,servicemonitor
# every guardrail and every observability hook already present — the template put them there

kubectl -n ledger-reports get rollout ledger-reports -w
# 10% -> 50% -> 100%, the exact canary shape from Part 2, with no one hand-configuring it

kubectl auth can-i delete secrets -n ledger-reports --as=system:serviceaccount:ledger-reports:ledger-reports
# no — scoped from birth, the same as ledger's Role earlier on this page

If any one of those checks fails, the platform isn't finished, not the exercise — go back and add the missing piece to the template until scaffolding a service really does produce all of it, automatically, every time.

What "done" looks like for the whole capstone

☺ Like you're 10: A cluster that fixes itself, ships safely, grows new attractions by itself, watches for trouble, and now guards every gate — six chapters, one finished park.

At the end of this part your platform-dev cluster carries everything all six parts built: a self-reconciling GitOps core (Part 1), a real pipeline and canary delivery (Part 2), a self-healing platform API (Part 3), a self-service portal that scaffolds fully-wired services (Part 4), a working observability stack with a triaged incident behind it (Part 5), and now — this page — policy-as-code in enforce mode, scoped RBAC, default-deny networking, mesh mTLS, a read supply-chain scan, fair multi-tenant quotas across two namespaces with visible cost, and proof that one self-service action produces all of the above with zero hand-written infrastructure. That is the golden path the entire course has been building toward, and you built it with your own hands.

Milestones

☺ Like you're 10: Tick each box only once you've actually watched it happen on your own screen — including the very last one, which is the whole capstone proving itself.

Work these in order — each depends on the cluster and repo state from the one before, and from every part before this one. Progress saves in this browser.

That's the capstone, start to finish. Part 6 added the guardrails and, more importantly, proved every earlier part's work compounds into one automatic golden path. Revisit Security & Policy Enforcement for the concepts behind everything on this page, step back to the Capstone Hub to see how all six parts fit together, or retrace the whole build from Part 1 — Foundation through Part 5 — Observability. When you're ready, take the muscle memory into the Exam Guide and the exam-prep checklist.