Capstone Part 3 — Networking & Ingress
This is the third of five parts building one continuous project: a real storefront running on Kubernetes, deployed the hard way so every mechanism is something you typed and watched work, not something a course video skipped past. Part 1 gave the cluster its shape — nodes, namespaces, and the storefront namespace this whole capstone lives in. Part 2 gave it something to run — the web and api Deployments, each with its own ConfigMap and Secret, each already passing its own liveness and readiness probes. Both have been reachable only from inside the cluster this whole time, one kubectl port-forward away from anyone who wanted to look. Today that changes, deliberately and in layers. This page gives the app stable Service identities, a public hostname with a browser-trusted TLS certificate that renews itself, and — the part most tutorials skip — a NetworkPolicy posture that starts from deny-everything and opens exactly four narrow, named holes. By the end, shop.example.com answers over HTTPS, and nothing in the cluster can reach api except web, on purpose.
Picture an apartment building that, until today, had no lock on the front door and no way for anyone outside to get in at all. First, every unit gets a proper buzzer with a name on it — press 4B and only 4B answers, no matter which actual tenant is home this week — that's a Service. Then the building gets one real, guarded front door with a working lock, a mailbox that only opens with a key nobody can forge, and a sign out front with the building's real, verified address — that's the Ingress and its certificate. Last, and most important: the building manager changes every door's lock to open for nobody by default, then hands out exactly the keys that make sense — the front door can reach the lobby, the lobby can reach 4B, and nothing else opens for anyone, no exceptions, until someone deliberately cuts a new key.
Arriving: a storefront namespace with web and api Deployments running healthy Pods, reachable only via kubectl port-forward or a shell inside the cluster — no Service, no public entry point, no NetworkPolicy at all, which on a stock cluster means every Pod can already reach every other Pod. Leaving this page: web and api Services with stable ClusterIPs, a storefront Ingress terminating TLS for shop.example.com off a cert-manager-issued certificate that renews itself, and a default-deny-all NetworkPolicy with exactly four targeted allows layered on top of it. Part 4 picks up exactly here and gives the api Pod somewhere durable to keep its data; Part 5 comes back to this same namespace to lock down who's even allowed to change any of it.
What this part assumes, and what it produces
☺ Like you're 10: The building, the units, and the tenants already exist — today is only about the front door, the mailboxes, and the locks.
This part assumes Part 2's web and api Deployments already exist and are healthy in the storefront namespace — kubectl get pods -n storefront should show every Pod Running with READY 1/1 before you touch anything below. You'll also need ingress-nginx and cert-manager installed as cluster add-ons — both are single Helm installs, covered in full on their own tool pages, and neither is specific to this capstone. Nothing here touches the Deployments, ConfigMaps, or Secrets Part 2 built; every object on this page is new, additive, and namespaced to storefront or cluster-scoped in a way that doesn't collide with anything that already exists. For the concepts underneath everything below — the flat, no-NAT Pod network, why NetworkPolicy is deny-by-selection rather than deny-by-default globally, and the exact Service/Ingress manifests this page builds on — see Services & Networking, the CKA blueprint domain this whole page is really a hands-on rep of.
The world model this part adds
| Thing | Name | Introduced |
|---|---|---|
| The application | storefront namespace: web + api Deployments | Part 1–2 |
web / api Services | web (ClusterIP :80→:8080), api (ClusterIP :8080→:8080) | Part 3 — this page |
| TLS issuer + certificate | letsencrypt-prod ClusterIssuer, shop-tls Certificate/Secret | Part 3 — this page |
| Public entrypoint | storefront Ingress, class nginx, host shop.example.com | Part 3 — this page |
| Traffic-shape policies | default-deny-all, allow-ingress-to-web, allow-web-to-api | Part 3 — this page |
| Egress-shape policies | allow-dns-egress, allow-api-egress-payments | Part 3 — this page |
Services: giving web and api a name that outlives any one Pod
☺ Like you're 10: Pods get replaced constantly and their addresses change every time — a Service is the one buzzer number that never changes, no matter which actual tenant answers it today.
Neither Deployment has a stable network identity yet — a Pod's IP is assigned when it starts and gone the moment it's rescheduled, which happens routinely. A Service fixes that with one virtual IP and DNS name in front of a changing set of Pods, selected purely by label; membership is tracked automatically in EndpointSlice objects that kube-proxy on every node watches and turns into real packet-forwarding rules, exactly as the API-and-controllers model and Networking & the CNI cover in depth. Both Services here are plain ClusterIP — internal-only is correct today, since nothing outside the cluster should ever reach either one directly; that's the Ingress's job, next section.
apiVersion: v1
kind: Service
metadata:
name: web
namespace: storefront
spec:
selector: { app: web }
ports:
- port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api
namespace: storefront
spec:
selector: { app: api }
ports:
- port: 8080
targetPort: 8080Apply both, then confirm each has real endpoints — not just a ClusterIP — before moving on: kubectl get endpointslices -n storefront -l kubernetes.io/service-name=web should list the actual Pod IPs behind it. A Service with a valid ClusterIP but zero endpoints is the single most common silent failure in this domain: the object exists, DNS resolves it, and every connection to it just times out, because its selector doesn't actually match any Pod's labels.
kubectl apply never validates that a Service's selector matches anything real; app: web against Pods labeled app: webapp applies cleanly, resolves in DNS, and answers every connection with a silent timeout instead of a helpful error. Before debugging anything downstream of a Service, always check kubectl get endpointslices -n storefront -l kubernetes.io/service-name=<svc> first — an empty list means the selector, not the network, is the bug.
The public entrypoint: Ingress, and a certificate that renews itself
☺ Like you're 10: The Ingress is the front door with a real lock; cert-manager is the helper who quietly swaps in a fresh ID card for that door every month, weeks before the old one expires, without anyone asking.
With both Services in place, one ClusterIssuer gives the whole cluster a single, shared way to get browser-trusted certificates from Let's Encrypt — create it once, and every namespace can reference it by name, exactly as cert-manager covers in full:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: platform-team@acme.io
privateKeySecretRef:
name: letsencrypt-prod-account-key
solvers:
- http01:
ingress:
ingressClassName: nginxNext, an explicit Certificate — written out in Git next to the Ingress that consumes its Secret, rather than left to the cert-manager.io/cluster-issuer ingress-shim annotation to create implicitly. Both routes end at the same object; writing it explicitly is what cert-manager's own checkpoint calls out as the better default for anything running in production, because the relationship between hostname, issuer, and Secret stays visible and reviewable in a pull request instead of hiding inside an annotation:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: shop-tls
namespace: storefront
spec:
secretName: shop-tls
duration: 2160h # 90d — ignored by Let's Encrypt, requested anyway
renewBefore: 720h # renew 30d before expiry
dnsNames:
- shop.example.com
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuerFinally, the Ingress itself — one object naming both routes and the certificate that protects them:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: storefront
namespace: storefront
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
ingressClassName: nginx
tls:
- hosts: [shop.example.com]
secretName: shop-tls
rules:
- host: shop.example.com
http:
paths:
- path: /api
pathType: Prefix
backend:
service: { name: api, port: { number: 8080 } }
- path: /
pathType: Prefix
backend:
service: { name: web, port: { number: 80 } }Apply all three in order — ClusterIssuer, then Certificate, then Ingress — and watch kubectl get certificate -n storefront shop-tls -w until READY flips to True. Behind that flip is the full ACME dance: a CertificateRequest, an Order, and an HTTP-01 Challenge that cert-manager solves through the Ingress it's already watching — the complete chain, and why evaluation can take a minute or two, is diagrammed on the cert-manager page itself; this page only needs the last mile.
Longest-matching-path wins, not YAML order — /api and / above are written with the more specific rule first purely for human readability, not because the Ingress spec cares which line comes first. If / were listed above /api in the same object, a request for /api/orders would still route to the api Service, because /api is the longer matching prefix. See Services & Networking and ingress-nginx for the full matching rules, including what happens when two separate Ingress objects claim the same host.
Default-deny first, then four narrow allows
☺ Like you're 10: The building manager changes every lock to open for nobody, then hands out exactly the keys that make sense — front door to lobby, lobby to 4B, nothing else, no exceptions, until someone deliberately cuts a new key.
Right now, with zero NetworkPolicy objects in storefront, the flat Pod network means every Pod in the cluster can already reach both web and api directly — a Pod in an unrelated namespace, a compromised sidecar three teams over, anything. That's the default Kubernetes ships with, and it's the opposite of what a public-facing app with a real backend should allow. The fix isn't one policy — it's five, applied in a specific order of reasoning even though Kubernetes evaluates them all at once: first, deny everything for every Pod in the namespace; then, name exactly what's still allowed to happen, one narrow rule at a time.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: storefront
spec:
podSelector: {}
policyTypes: [Ingress, Egress]An empty podSelector: {} matches every Pod in the namespace; naming both Ingress and Egress in policyTypes with no ingress or egress rules underneath means every Pod here now accepts nothing in and sends nothing out — including, easy to forget, DNS lookups. Apply this alone and both web and api go instantly, completely dark. That's expected, and it's exactly why the four policies below exist.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-ingress-to-web
namespace: storefront
spec:
podSelector: { matchLabels: { app: web } }
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector:
matchLabels: { kubernetes.io/metadata.name: ingress-nginx }
podSelector:
matchLabels: { app.kubernetes.io/name: ingress-nginx }
ports:
- { port: 8080, protocol: TCP }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-web-to-api
namespace: storefront
spec:
podSelector: { matchLabels: { app: api } }
policyTypes: [Ingress]
ingress:
- from:
- podSelector: { matchLabels: { app: web } }
ports:
- { port: 8080, protocol: TCP }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns-egress
namespace: storefront
spec:
podSelector: {}
policyTypes: [Egress]
egress:
- to:
- namespaceSelector:
matchLabels: { kubernetes.io/metadata.name: kube-system }
podSelector:
matchLabels: { k8s-app: kube-dns }
ports:
- { port: 53, protocol: UDP }
- { port: 53, protocol: TCP }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-api-egress-payments
namespace: storefront
spec:
podSelector: { matchLabels: { app: api } }
policyTypes: [Egress]
egress:
- to:
- ipBlock: { cidr: 203.0.113.0/24 } # payments provider's published range
ports:
- { port: 443, protocol: TCP }Read allow-ingress-to-web's single from entry carefully — namespaceSelector and podSelector sit inside the same list item, which means they combine with AND: only Pods matching both selectors at once. Split them into two separate entries in the from array instead and the meaning changes completely, to OR — any Pod in the ingress-nginx namespace regardless of its own labels, or any Pod anywhere carrying that controller's label. The single-entry, AND'd form here is deliberately the narrower one: only the actual ingress-nginx controller Pod, in its own namespace, gets in.
allow-api-egress-payments allows traffic to 203.0.113.0/24 — not to "the payments provider," even though that's what it means today. NetworkPolicy has no concept of a hostname; if that provider rotates or expands its published IP range and nobody updates the CIDR here, the egress call starts failing with a NetworkPolicy silently in the way, and the failure looks exactly like a downstream outage rather than a stale allow-list. Treat any ipBlock rule against a third party's range as something to revisit on a schedule, the same way cert-manager's page treats a certificate's expiry — a thing that's correct today and silently wrong later unless someone owns re-checking it.
NetworkPolicies for the same Pod are always additive, never a stricter intersection — allow-web-to-api and any future policy that also selects app: api both apply at once, and the union of everything any of them allows is what gets through. That's why default-deny-all has to exist before any of this reasoning holds: without it, there's no baseline to add narrow exceptions on top of — every Pod starts open, and a NetworkPolicy can only ever take traffic away from an already-selected Pod, never restrict a Pod nothing has selected yet.
"I don't carry a single packet myself — I just make sure the address you asked for lands on the Pod that's actually supposed to answer right now, even though that Pod didn't exist five minutes ago. Timmy's rules don't slow me down at all when they're right; they only ever stop the requests that were never supposed to reach that Pod in the first place. The two of us aren't in tension — a Service that finds the right Pod and a NetworkPolicy that guards it are solving completely different problems that happen to sit on the exact same wire."
Proving it: what should work, and what absolutely should not
☺ Like you're 10: Ring the buzzer from outside — it should work. Try walking straight into 4B from the hallway without buzzing anyone — it shouldn't, and now it can't.
Apply everything above in order — Services, then the ClusterIssuer/Certificate/Ingress, then all five NetworkPolicies — and prove both halves independently. First, the app is genuinely reachable:
curl -sv https://shop.example.com/ 2>&1 | grep -E "HTTP/|subject:|issuer:"
# HTTP/2 200
# subject: CN=shop.example.com
# issuer: C=US, O=Let's Encrypt, CN=R3
# -- a REAL issuer, not cert-manager's self-signed fallback cert
curl -s https://shop.example.com/api/healthz
# {"status":"ok"} -- routed through web's Ingress rule to the api Service, over the policy chainThen prove the negative — the part a passing curl above can't show on its own. From a throwaway Pod with no matching label, in a completely different namespace:
kubectl run stray --rm -it --restart=Never --image=curlimages/curl -n default -- \
curl -m 3 http://api.storefront.svc.cluster.local:8080/healthz
# curl: (28) Connection timed out after 3001 milliseconds
# -- default-deny-all blocks it; allow-web-to-api only names Pods labeled app: web, and this isn't one
kubectl run stray --rm -it --restart=Never --image=curlimages/curl -n storefront -- \
curl -m 3 http://web.storefront.svc.cluster.local:80/
# curl: (28) Connection timed out after 3001 milliseconds
# -- same namespace isn't enough either: allow-ingress-to-web only names the ingress-nginx
# controller as a source, and this Pod carries neither its namespace nor its labelBoth timeouts are the point, not a bug to chase. The first curl succeeding and both of these failing, together, is what "default-deny with targeted allows" actually means in practice — reachable exactly where it should be, and nowhere else, proven from the outside rather than assumed from reading YAML.
The Kubernetes API accepts a NetworkPolicy object whether or not anything in the cluster enforces it. Calico, Cilium, and most managed cloud CNIs implement the NetworkPolicy API; plain Flannel by default does not — the object saves cleanly, kubectl get networkpolicy lists it, and it silently does nothing. If the "should be blocked" curl above succeeds instead of timing out, this — not a typo in the YAML — is the first thing to check. See Calico and Cilium for the two CNIs that do enforce it, and Networking & the CNI for why the CNI spec leaves enforcement optional in the first place.
Milestones
☺ Like you're 10: Tick each box only once you've actually watched it happen on your own screen, not because the step "sounds right."
Work these in order — each depends on the state left by the one before. Progress saves in this browser.
web and api Servicesstorefront namespace.kubectl get endpointslices -n storefront shows real Pod IPs behind both.letsencrypt-prod ClusterIssuerkubectl describe clusterissuer letsencrypt-prod shows condition Ready: True.Certificate and the storefront Ingresskubectl get certificate -n storefront shop-tls shows READY True, SECRET shop-tls.curl -sv https://shop.example.com/, reading the certificate's issuer: line.200 and the issuer is Let's Encrypt, not cert-manager's self-signed fallback.default-deny-all, and watch the site go darkallow-ingress-to-webweb from an unrelated Pod still times out.allow-web-to-apicurl https://shop.example.com/api/healthz returns 200 through the full chain, not just directly against the api Pod.allow-dns-egresskubectl exec into any storefront Pod and nslookup kubernetes.default resolves.allow-api-egress-paymentsapi Pod succeeds, while a curl to any other external IP from that same Pod still times out.ipBlock caveatkubectl run stray ... commands from the proving section, unmodified.Pip the Hummingbird: Buzzer's up, lock's on the door, real certificate — shop.example.com is live and it's actually us answering.
Timmy the Turtle: And nothing else can walk in the back. I checked — twice, from two different directions.
Benny the Beaver: QA just pinged me — a debug Pod they spun up in default can't reach api at all, not even for a health check.
Gizmo the Gremlin: Easy fix — just delete default-deny-all. Back to everything-reaches-everything, ship it, nobody will notice. 😈
Timmy: Absolutely not. That debug Pod was never supposed to reach api — that's the policy working, not breaking. If QA needs it, they get their own narrow allow scoped to their own Pod's label, same as everyone else got.
Nutty the Squirrel: I'll file that one — allow-qa-debug-to-api, port 8080 only, expires when the debug Pod does.
Benny: ...that's fair. A five-minute review beats an outage from a hole nobody remembers opening.
1. Why does default-deny-all have to exist before any of the four "allow" policies actually mean anything? 2. In allow-ingress-to-web, why does putting namespaceSelector and podSelector in the same from entry matter, and what would change if they were two separate entries in the array instead? 3. Why does allow-dns-egress target the kube-system namespace with the k8s-app: kube-dns podSelector specifically, rather than a bare namespaceSelector: {} matching every namespace? 4. What's the real risk in using an ipBlock CIDR to allow api's egress to the payments provider, instead of something DNS-based? 5. Walk through, layer by layer, why the second stray-Pod curl in the proving section — from inside storefront itself — still times out. 6. If the "should be blocked" curl from a stray Pod unexpectedly succeeds, what should you check before assuming your YAML has a bug?
Check your answers
- Without it, every Pod in the namespace starts fully open by default — a NetworkPolicy can only take traffic away from a Pod some policy has already selected for deny; it can never restrict a Pod that no policy touches.
default-deny-allis what makes every later "allow" mean something, by giving them a closed baseline to open narrow exceptions on top of. - Selectors inside the same
fromlist entry combine with AND — only a Pod matching both the namespace and the pod label gets through, which here means only the real ingress-nginx controller Pod. Splitting them into two separate entries changes the meaning to OR: any Pod in that namespace regardless of its labels, or any Pod anywhere carrying that label — a much wider, and much less intentional, hole. - A bare
namespaceSelector: {}would allow egress to every Pod in every namespace on port 53, not just to CoreDNS — a far wider hole than "let this Pod resolve DNS" actually requires. Naming the specific namespace and the CoreDNS Pod's own label keeps the allow scoped to exactly the one destination the rule is meant to permit. - NetworkPolicy has no concept of a hostname — an
ipBlockrule allows traffic to a fixed CIDR, not to "whoever currently owns that CIDR." If the payments provider rotates or expands its published IP range and the rule isn't updated, egress starts failing with the NetworkPolicy silently in the way, and the failure looks exactly like a downstream outage rather than a stale allow-list needing a refresh. - Being in the same namespace isn't a NetworkPolicy exemption by itself.
allow-ingress-to-webonly names the ingress-nginx controller (by namespace and label) as an allowed source forweb; a stray Pod insidestorefrontcarrying neither that namespace nor that label matches no rule at all, sodefault-deny-all's baseline still applies to it and the connection times out. - Confirm the CNI actually implements the NetworkPolicy API before trusting the object did anything — the Kubernetes API server accepts and stores a NetworkPolicy whether or not anything enforces it, and a CNI like plain Flannel that doesn't implement the API will let every policy on this page silently do nothing while
kubectl get networkpolicystill lists them as if they were active.
Part 3 gave the storefront a real front door, a certificate that renews itself, and a network posture that denies everything until you've said otherwise. Continue to Capstone Part 4 — Storage & Stateful Apps, where api gets somewhere durable to keep its data behind everything built here. Or step back to Build a Cluster — Start Here for how all five parts fit together, revisit Services & Networking and Networking & the CNI for the concepts behind what you just built, cross-check the security framing against Security: Defense in Depth and, cross-course, DevSecOps's Kubernetes security deep dive, or drill this exact failure mode under timed conditions in Drill — Diagnose a Networking Failure.