Networking & Service Connectivity

Every platform is, underneath, a network. Before a developer’s service can talk to a database, before a user’s request can reach a pod, before two clusters can share a workload, packets have to find their way — and Kubernetes deliberately doesn’t ship that plumbing in the box. It defines a model and a set of extension points, then trusts you to fill them with the right pieces: a CNI, a Service proxy, a DNS server, an ingress path, maybe a mesh. This page is the deep tour of that stack — how a packet travels from one pod to another, how a stable name resolves to a moving target, how traffic gets in and safely back out, and how it all stretches across clusters. Get this layer right and the rest of the platform feels like magic; get it wrong and every outage traces back here.

The Kubernetes network model

☺ Like you’re 10: Kubernetes doesn’t build the roads itself — it hands you a rulebook that says “every house needs its own address and any house must be able to reach any other,” then lets you pick who paves the roads.

Kubernetes is famously unopinionated about networking. Rather than bundle a specific implementation, it publishes a small set of hard requirements — the Kubernetes network model — and delegates the actual wiring to a pluggable component (the CNI, next section). Any implementation that satisfies the model is a valid Kubernetes network. The model has three non-negotiable rules: every Pod gets its own unique, cluster-wide IP address; every Pod can communicate with every other Pod on any node without NAT; and every node’s agents (the kubelet, system daemons) can reach the Pods on that node. This is usually summarised as “IP-per-Pod on a flat network.”

IP-per-Pod and the shared network namespace

A Pod is not a single container — it’s a small group of containers that share fate and, crucially, share a network namespace. When the kubelet starts a Pod it first creates an almost-empty “sandbox” (historically the pause container) that owns the Pod’s network namespace and holds the Pod’s IP. Every application container in the Pod then joins that same namespace. The practical consequence is that containers inside one Pod see each other on localhost and share the same set of ports — an app on :8080 and a sidecar proxy on :15001 live in the same loopback world. This is exactly why a service-mesh sidecar can transparently intercept a Pod’s traffic: it’s literally in the same network namespace.

☺ Like you’re 10: All the containers in one Pod share one “phone line.” They talk to each other by just picking up the phone (localhost), and to the outside world they share one phone number (the Pod IP).

A flat network with no NAT — and why that matters

The “no NAT between Pods” rule is the quiet hero of the whole design. On a flat network, when Pod A at 10.244.1.4 talks to Pod B at 10.244.2.7, Pod B sees the connection as coming from 10.244.1.4 — the real source, unmasqueraded. That single guarantee erases a mountain of pain that plagued older container systems: there are no port-mapping collisions (every Pod has the full port range to itself), service discovery is simple (a Pod IP is a real, routable thing), and security policy can be written against actual source identities instead of a soup of translated addresses. The cost is that something has to make a flat address space work across many machines with separate physical networks — and that “something” is the CNI plugin.

The four connectivity problems

It helps to see the whole model as four distinct problems, each solved by a different layer. This map is the scaffolding for the rest of the page:

CNI — the Container Network Interface

☺ Like you’re 10: The CNI is the road-building crew. When a new Pod is born, the crew shows up, gives it an address, and connects its little driveway to the big flat road so it can reach everyone else.

The Container Network Interface is a tiny, deliberately minimal specification: a contract between the container runtime and a networking plugin. When the kubelet asks the runtime to start a Pod’s sandbox, the runtime invokes the configured CNI plugin with a verb — ADD to wire the Pod up, DEL to tear it down, CHECK to verify. The plugin is responsible for creating the Pod’s network interface (typically one end of a veth pair, with the other end in the node’s root namespace), assigning it an IP via its IPAM (IP Address Management) module, and programming routes so the flat model holds. Because the interface is so small, an entire ecosystem of plugins competes on how they implement the “flat network” underneath.

What a CNI plugin actually does

Concretely, on Pod creation a CNI plugin: (1) allocates a free IP from the node’s Pod CIDR (a slice of the cluster’s Pod address space carved out per node); (2) creates a veth pair, moving one end into the Pod’s namespace as eth0 and leaving the other on the host; (3) installs routing so packets to that Pod IP land in the right namespace; and (4) records the allocation so it isn’t handed out twice. On delete, it reverses all of that and returns the IP to the pool. Everything else — how a packet gets from a Pod on Node A to a Pod on Node B — is where implementations diverge, and that choice is the single biggest networking decision a platform team makes.

Overlay vs BGP — two ways to make “flat” real

Nodes usually sit on a physical network that knows nothing about Pod IPs. There are two dominant strategies for bridging that gap. An overlay network encapsulates each Pod packet inside a node-to-node packet (commonly VXLAN or Geneve): the Pod packet becomes the payload, wrapped with an outer header addressed node-to-node, unwrapped on arrival. Overlays work almost anywhere because the physical network only ever sees ordinary node-to-node traffic — but encapsulation adds CPU cost and shrinks the usable MTU, so you must account for the extra header bytes or suffer mysterious fragmentation. The alternative is native routing, usually via BGP: each node advertises “I own Pod CIDR 10.244.1.0/24” to the network fabric (or to a route reflector), so Pod packets are routed natively with no encapsulation. BGP is faster and simpler on the wire, but it needs the underlying network (or your cloud’s routing) to cooperate.

☺ Like you’re 10: Overlay is like putting your letter inside a second envelope so the postal system only sees the outer one — easy, works everywhere, but heavier. BGP is like teaching the whole post office your street exists, so your letter goes straight there — faster, but only if the post office will listen.

The field: Flannel, Calico, and Cilium

Three names dominate. Flannel is the minimalist: a simple VXLAN overlay that just delivers the flat network, with essentially no policy engine. It’s easy to reason about and a fine choice for small or learning clusters, but it stops where security and observability begin. Calico pairs a high-performance datapath (native BGP routing by default, overlay when needed) with a mature NetworkPolicy engine, and it offers an eBPF dataplane mode as an alternative to iptables. Cilium is the modern heavyweight: its datapath is built on eBPF (programs attached directly into the Linux kernel), giving it identity-based policy, L7-aware filtering, kube-proxy replacement, rich flow observability (Hubble), and Cluster Mesh — all without the per-packet cost of long iptables chains. Cilium is why many platform teams now treat the CNI as a strategic platform component rather than a checkbox.

Who enforces NetworkPolicy

A subtlety that trips up newcomers: the Kubernetes API accepts a NetworkPolicy object, but the API server does not enforce it — the CNI does. If your CNI has no policy engine (plain Flannel), your policies are silently inert, which is a dangerous false sense of security. The model is also default-allow until you select a Pod: with no policies, all traffic flows. The moment a policy selects a Pod for a direction (ingress or egress), that direction flips to default-deny for that Pod, and only the explicitly allowed traffic passes. A classic hardening pattern is a namespace-wide default-deny, then additive allow rules. This is where networking meets Security & Policy Enforcement — micro-segmentation is a networking control with a security purpose.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: checkout-allow
  namespace: shop
spec:
  podSelector:
    matchLabels: { app: checkout }   # selecting a Pod flips it to default-deny
  policyTypes: [Ingress, Egress]
  ingress:
    - from:
        - podSelector:
            matchLabels: { app: web }   # only the web tier may call checkout
      ports:
        - protocol: TCP
          port: 8080
  egress:
    - to:
        - podSelector:
            matchLabels: { app: payments }
      ports:
        - { protocol: TCP, port: 8443 }
    - to:                               # allow DNS so the Pod can resolve names
        - namespaceSelector: {}
          podSelector:
            matchLabels: { k8s-app: kube-dns }
      ports:
        - { protocol: UDP, port: 53 }
        - { protocol: TCP, port: 53 }

Services & kube-proxy

☺ Like you’re 10: Pods come and go and their addresses change. A Service is a permanent front-desk number that never changes — you call it, and it quietly forwards you to whichever Pod is healthy right now.

Pods are cattle: they’re created, killed, and rescheduled constantly, and each new Pod gets a fresh IP. You can’t hardcode a Pod IP and expect it to survive a rollout. A Service solves this by providing a stable virtual identity — a ClusterIP that never changes for the life of the Service — in front of a dynamic set of backing Pods selected by label. Something has to translate “traffic to the virtual IP” into “traffic to a real, healthy Pod,” and that something is the Service dataplane, classically implemented by kube-proxy.

The Service types

There are five shapes, each layering on the last:

The mental model: ClusterIP is the base, NodePort and LoadBalancer are additive ways to expose that same base outward, Headless opts out of virtual-IP load balancing (essential for StatefulSets and databases where each replica has an identity), and ExternalName is a pure DNS alias — a graceful way to point db.internal at an RDS hostname without changing app config.

apiVersion: v1
kind: Service
metadata: { name: checkout, namespace: shop }
spec:
  selector: { app: checkout }
  ports:
    - { name: http, port: 80, targetPort: 8080 }
  # type defaults to ClusterIP — a stable virtual IP for the whole cluster
---
apiVersion: v1
kind: Service
metadata: { name: cassandra, namespace: data }
spec:
  clusterIP: None            # HEADLESS — DNS returns each Pod IP, no VIP
  selector: { app: cassandra }
  ports:
    - { port: 9042 }

EndpointSlices — the moving target, tracked

Behind every Service is the live list of “which Pod IPs are currently ready to receive traffic.” Originally this lived in a single Endpoints object per Service — which became a scaling disaster: a Service with thousands of Pods produced one enormous object that was rewritten (and re-sent to every node) on every Pod change. EndpointSlices fixed this by sharding that list into chunks of (by default) up to 100 endpoints, so a single Pod flapping only rewrites one small slice. The EndpointSlice controller watches Pods and readiness, keeps the slices current, and kube-proxy (or the eBPF dataplane) watches the slices to program the dataplane. Slices also carry richer per-endpoint data — topology hints for topology-aware routing (prefer a same-zone backend to cut cross-zone cost and latency), and per-endpoint conditions.

The dataplane — iptables vs IPVS vs eBPF

How the virtual-IP-to-Pod translation is actually programmed matters enormously at scale. There are three approaches. The traditional kube-proxy mode uses iptables: each Service becomes a chain of rules, and a packet to a ClusterIP is DNAT-ed to a randomly chosen backend endpoint. It’s battle-tested but scales poorly — rule evaluation is roughly linear, so tens of thousands of Services mean long chains and slow updates. IPVS mode uses the kernel’s in-kernel L4 load balancer with hash-table lookups (near-constant time) and real load-balancing algorithms (round-robin, least-connection). The modern option is an eBPF dataplane (Cilium, or Calico’s eBPF mode) that replaces kube-proxy entirely: Service translation happens in eBPF programs in the kernel with hash-map lookups, no iptables chains, faster updates, and better observability.

DNS & service discovery

☺ Like you’re 10: DNS is the cluster’s phone book. You remember the name “checkout,” and the phone book tells you the number to actually dial — even after the number changes.

Stable virtual IPs are still IPs, and nobody wants to configure applications with numbers. CoreDNS is the cluster’s DNS server: a small, pluggable DNS engine that runs as a Deployment (fronted, fittingly, by a Service), and every Pod is configured to send DNS queries to it. CoreDNS turns Service and Pod names into addresses, which is what makes checkout.shop.svc.cluster.local a thing you can actually connect to. It’s the discovery layer that lets everything else stay loosely coupled.

CoreDNS and the plugin chain

CoreDNS is configured by a Corefile — an ordered chain of plugins that each query flows through. The star is the kubernetes plugin, which watches the API server for Services and EndpointSlices and answers cluster-domain queries directly from that live data. Other plugins add caching, forwarding of external names to an upstream resolver, health, and metrics. Because it’s just a chain, platform teams can extend DNS behaviour — rewrite rules, split-horizon views, per-domain forwarding to a corporate resolver — without swapping the whole system.

# CoreDNS Corefile (ConfigMap coredns in kube-system)
.:53 {
    errors
    health
    kubernetes cluster.local in-addr.arpa ip6.arpa {
        pods insecure
        fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153        # golden-signal metrics for the DNS tier itself
    forward . /etc/resolv.conf   # anything not cluster.local → upstream
    cache 30
    loop
    reload
}

# From any Pod:
$ nslookup checkout.shop.svc.cluster.local
Name:   checkout.shop.svc.cluster.local
Address: 10.96.0.10                 # the Service ClusterIP

The Service FQDN, search domains, and ndots

Every Service gets a fully-qualified name of the form <service>.<namespace>.svc.cluster.local, which resolves to the ClusterIP (an A/AAAA record); named ports also get SRV records. You rarely type the full name, because the kubelet writes a search list into every Pod’s /etc/resolv.conf — so a Pod in namespace shop can say just checkout (same namespace) or checkout.shop (any namespace) and the resolver appends the search suffixes to complete it. This convenience hides a classic performance gotcha: ndots. Kubernetes sets ndots:5, meaning any name with fewer than five dots is first tried against every search domain before being tried as-is. Calling an external name like api.stripe.com (two dots) therefore fires several failed cluster lookups before the real one succeeds — extra latency and DNS load. Ending external names with a trailing dot (api.stripe.com.) or tuning ndots per Pod avoids the storm.

Headless services & stable Pod DNS

Sometimes you don’t want a single virtual IP that hides the Pods — you want to see every Pod. A headless Service (clusterIP: None) makes CoreDNS return the individual Pod IPs (multiple A records) instead of a VIP, letting the client do its own selection. This is the backbone of stateful systems: a StatefulSet paired with a headless Service gives each Pod a stable, predictable DNS name like cassandra-0.cassandra.data.svc.cluster.local, so a database replica can be addressed by identity across restarts. Ordinary Pods can also be resolved by their (dash-encoded) IP under pod.cluster.local, though named endpoints via headless Services are what you actually build on. This is exactly the kind of identity-preserving networking that Storage & State relies on.

Getting traffic in — Ingress vs the Gateway API

☺ Like you’re 10: So far everyone inside the building can call each other. Now the outside world wants in — so we need a smart front door that reads the visitor’s request and sends them to the right room.

Everything above is east-west traffic — service to service, inside the cluster. Getting the outside world in is north-south traffic, and it needs an L7-aware front door that can terminate TLS and route by hostname and path. Historically that was the Ingress resource; the future is the Gateway API. Both describe intent declaratively and both need a controller to make them real, but they differ sharply in expressiveness and in who owns what.

Ingress and ingress controllers

The Ingress resource is a compact way to say “route shop.example.com/api to the api Service and /web to the web Service, terminating TLS with this certificate.” But the Ingress spec itself is minimal — it only really models host/path HTTP routing — so every feature beyond that (rewrites, rate limits, canary weighting, auth, timeouts) got bolted on through controller-specific annotations. An ingress controller — most commonly ingress-nginx, but also HAProxy, Traefik, or the cloud providers’ own — watches Ingress objects and configures the actual proxy. The result works, and ingress-nginx runs a huge share of the internet, but the annotation sprawl is non-portable: an Ingress tuned for nginx won’t behave the same on Traefik, and complex routing turns into a pile of vendor-specific strings.

The Gateway API role model

The Gateway API is the successor, designed to fix Ingress’s two biggest problems: limited expressiveness and blurred ownership. It splits the single Ingress resource into a small family of typed resources aligned to who is responsible. A GatewayClass is installed by the infrastructure provider (like a StorageClass — it names an implementation such as Istio, Cilium, or Envoy Gateway). A Gateway is created by the cluster/platform operator: it declares listeners (ports, protocols, TLS, allowed hostnames) and represents actual infrastructure — a load balancer and proxy. And a Route (HTTPRoute, GRPCRoute, TCPRoute…) is written by the application developer to attach their routing rules to a Gateway. This role separation is the whole point: the platform team owns the front door and its TLS, while app teams self-serve their routes without touching shared infrastructure.

☺ Like you’re 10: The building owner installs the front door and the lock (the Gateway). Each team just puts up a little sign inside — “deliveries for us go to room 12” (an HTTPRoute) — without ever touching the door itself.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway                     # owned by the PLATFORM team
metadata: { name: edge, namespace: infra }
spec:
  gatewayClassName: cilium
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      tls:
        mode: Terminate
        certificateRefs: [{ name: shop-tls }]
      allowedRoutes:
        namespaces: { from: Selector, selector: { matchLabels: { team: shop } } }
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute                   # owned by the APP team, in their namespace
metadata: { name: checkout, namespace: shop }
spec:
  parentRefs: [{ name: edge, namespace: infra }]   # attach to the shared Gateway
  hostnames: ["shop.example.com"]
  rules:
    - matches: [{ path: { type: PathPrefix, value: /checkout } }]
      backendRefs: [{ name: checkout, port: 80 }]

Why the Gateway API supersedes Ingress

Ingress is now effectively frozen — it will receive no new features — and the Gateway API is the community’s forward direction (its core is GA). The reasons are structural, not cosmetic:

That last row is a genuine self-service enabler: route delegation lets a platform team run one shared Gateway while app teams attach routes from their own namespaces, and a ReferenceGrant explicitly authorises any cross-namespace reference (say, a Route pointing at a backend elsewhere). It’s multi-tenant north-south routing without handing anyone the keys to the front door — a pattern that plugs straight into Self-Service & Developer Portals.

LoadBalancer services & MetalLB

A Gateway or a type: LoadBalancer Service both ultimately need a real external IP. In a cloud, the cloud controller manager provisions an actual cloud load balancer automatically. On bare metal there’s no cloud LB to call, so a LoadBalancer Service would sit forever in <pending>. MetalLB fills that gap: it hands out external IPs from a pool you define and advertises them to the local network in one of two modes — L2 mode (one elected node answers ARP/NDP for the IP; simple, but all traffic funnels through that node) or BGP mode (nodes peer with your routers so traffic is truly load-balanced across nodes, at the cost of needing BGP-capable network gear). It’s the piece that makes on-prem clusters feel cloud-like. This is core to Platform Architecture & Infrastructure when you run your own metal.

Service mesh

☺ Like you’re 10: A service mesh is like giving every program its own tiny bodyguard-translator who sits at the door, checks IDs, encrypts every message, retries dropped calls, and writes down exactly what happened — so the program itself doesn’t have to learn any of that.

Once you have dozens of services calling each other, a whole class of concerns shows up in every service: mutual TLS between services, retries and timeouts, circuit breaking, fine-grained traffic splitting, and consistent telemetry. Implementing all of that in each application, in every language, is a nightmare. A service mesh pulls those concerns out of the app and into the network layer, so they’re configured centrally and applied uniformly. It’s the platform’s answer to “make service-to-service communication secure, reliable, and observable by default.”

What a mesh gives you

Four capabilities justify the complexity. First, mTLS: the mesh issues each workload a cryptographic identity and encrypts + mutually authenticates all service-to-service traffic automatically — zero-trust networking without app changes, the networking half of what Security & Policy calls workload identity. Second, L7 traffic management: retries, timeouts, and circuit breaking (shed load from a failing dependency before it drags you down), plus weighted traffic splitting between versions — the mechanism behind the canaries and blue-green rollouts in CI/CD & Progressive Delivery. Third, observability: because every request passes through a proxy, you get consistent golden-signal metrics, distributed traces, and access logs for free. Fourth, authorization: policy on who may call whom, expressed in service identity rather than IP.

# Istio: require mTLS everywhere, then shift 10% of traffic to v2 (a canary)
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: { name: default, namespace: shop }
spec:
  mtls: { mode: STRICT }          # reject any non-mTLS traffic in this namespace
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata: { name: checkout, namespace: shop }
spec:
  hosts: [checkout]
  http:
    - route:
        - destination: { host: checkout, subset: v1 }
          weight: 90
        - destination: { host: checkout, subset: v2 }
          weight: 10           # progressive delivery, enforced in the mesh
      retries: { attempts: 3, perTryTimeout: 2s }
      timeout: 10s

Sidecar vs ambient (sidecarless)

The classic mesh architecture is the sidecar: a proxy (usually Envoy) is injected into every application Pod, sharing its network namespace and intercepting all traffic. It’s powerful and mature, but it has real costs — a proxy container per Pod means extra memory and CPU across the fleet, added per-hop latency, and lifecycle friction (the sidecar must start before the app and outlive it; upgrading the mesh means restarting every Pod). The newer answer is ambient (sidecarless) mesh, which splits the work by layer: a per-node ztunnel handles L4 and mTLS for all Pods on that node, and an optional per-namespace waypoint proxy handles L7 policy only where you actually need it. You pay for L7 processing solely where it’s used, and mesh upgrades no longer restart apps. Cilium’s mesh takes a related eBPF-based sidecarless route, doing much of the L4 work in the kernel and using a shared per-node Envoy for L7.

☺ Like you’re 10: Sidecar mode gives every program its own personal bodyguard (lots of bodyguards, lots of lunch money). Ambient mode hires one guard per floor for the basics, and only brings in a specialist when a room actually needs one.

Istio vs Linkerd vs Cilium Service Mesh

Three meshes lead the field, with different philosophies. Istio is the most feature-rich and configurable (Envoy-based, with both sidecar and ambient modes) — the Swiss-army knife, at the price of a steeper learning curve. Linkerd optimises for simplicity and speed: it uses a purpose-built, ultralight Rust micro-proxy (linkerd2-proxy) instead of Envoy, and prioritises operational ease and low overhead over maximal features. Cilium Service Mesh reuses your CNI’s eBPF datapath to deliver much of a mesh with fewer (or no) sidecars, which is attractive when Cilium is already your networking layer.

Multi-cluster & east-west networking

☺ Like you’re 10: One building is easy — everyone can call everyone. Now imagine two buildings across town that need to call each other as if they were one. You have to connect their phone systems and make sure no two rooms accidentally share a number.

Real platforms outgrow a single cluster — for blast-radius isolation, regional locality, or capacity. The moment you have two clusters that must cooperate, cross-cluster east-west networking becomes a first-class problem, and it’s meaningfully harder than the single-cluster case. This is the deep end of Multi-Cluster & Fleet Management.

Why multi-cluster networking is hard

Three problems bite immediately. Overlapping CIDRs: clusters are usually built independently with the same default Pod/Service ranges, so 10.244.0.0/16 in cluster A collides with the identical range in cluster B — Pod IPs are no longer globally unique, breaking the flat model across clusters unless you plan address space up front or use gateways that translate. Cross-cluster discovery: a Service name resolves only within its own cluster’s DNS, so checkout.shop.svc means nothing in the other cluster without a mechanism to export it. And identity & trust: mTLS depends on a shared trust domain, so two clusters must agree on a common root of trust before their workloads can mutually authenticate. Solve all three and “two clusters” can start to feel like “one big cluster.”

Cluster mesh

A cluster mesh stitches clusters into a unified network and service space. Cilium Cluster Mesh, for example, gives Pods in different clusters direct connectivity and makes Services global: a Service with the same name in multiple clusters can be treated as one, with traffic preferring local endpoints and failing over to a remote cluster only when the local backends are gone. Istio does the analogous thing at the mesh layer via multi-primary or primary-remote topologies, extending mTLS identity and traffic policy across cluster boundaries. Either way you get cross-cluster load balancing, locality-aware routing (stay in-region to cut latency and egress cost), and failover — the networking substrate for genuinely resilient, multi-region platforms.

Multi-Cluster Services (the MCS API)

To standardise this, the community defined the Multi-Cluster Services (MCS) API. You ServiceExport a Service in one cluster to publish it to the clusterset; a corresponding ServiceImport appears in the other clusters, resolvable under the clusterset.local domain. It’s the portable, implementation-agnostic contract for “this Service is available fleet-wide,” letting workloads discover and reach services across clusters without hardcoding remote addresses.

# In cluster A: publish "checkout" to the whole clusterset
apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceExport
metadata:
  name: checkout            # matches the Service name
  namespace: shop
# → an implementation creates a matching ServiceImport in the other clusters,
#   resolvable as:  checkout.shop.svc.clusterset.local

Egress, load balancing & the edge

☺ Like you’re 10: We spent all this time letting people in. Now we also have to watch who goes out — and make sure visitors from the internet land on a healthy room even if one whole building loses power.

The last mile of platform networking is the edge: controlling outbound (egress) traffic, balancing inbound (north-south) traffic across nodes and regions, and steering users to healthy locations with DNS. These are where networking, security, and reliability meet.

Egress control

By default, Pods can reach anything on the internet — which is a data-exfiltration risk and a compliance headache. Locking down egress is as important as locking down ingress. The base tool is an egress NetworkPolicy (default-deny outbound, then allow only the destinations a workload legitimately needs). Because IPs are a poor way to name SaaS endpoints, richer setups use DNS-aware egress (allow by hostname, e.g. Cilium’s FQDN policies) or route all outbound traffic through an egress gateway — a controlled set of nodes with fixed, known source IPs — so partners can allow-list you and security can audit every byte that leaves. Egress control is a shared concern with Security & Policy Enforcement: the network is your last chance to stop data walking out the door.

North-south load balancing and source IPs

External traffic entering the cluster is balanced by a load balancer at L4 (transport) or L7 (application). One detail bites hard in practice: source IP preservation. A LoadBalancer Service has an externalTrafficPolicy: with Cluster (the default), traffic may be forwarded to a Pod on another node, which requires SNAT and therefore hides the client’s real IP — but spreads load evenly. With Local, traffic only goes to Pods on the node that received it, which preserves the client IP (vital for allow-lists, geo-IP, rate limiting, and audit logs) and cuts an extra hop, at the cost of possible imbalance if Pods aren’t evenly spread. Knowing which you need — and why an audit log full of node IPs means you picked wrong — is exactly the kind of failure mode this layer hides.

Global load balancing at the edge

Above any single cluster’s load balancer sits global / DNS-based load balancing (GSLB) for multi-region resilience and latency. Instead of one IP, a name like shop.example.com resolves — via a smart, health-checked authoritative DNS layer — to whichever regional endpoint is healthy and closest to the user, steering by geography, latency, or weight and pulling a region out of rotation when its health checks fail. Frequently this is paired with anycast, where the same IP is advertised from many locations and the internet’s own routing delivers each user to the nearest one. Together they’re how a platform survives losing an entire region without users noticing — the edge complement to the in-cluster reliability work, and a reminder that platform networking ultimately reaches all the way out to the client’s resolver.

Networking is the platform’s nervous system — invisible when it works, catastrophic when it doesn’t. From the flat pod network up through Services, DNS, the Gateway API, the mesh, and out across clusters to the edge, every layer trades a little complexity for a lot of decoupling, so that Dot can ship a feature by typing a name instead of chasing an IP. Next, follow the wire down to the machines it runs on in The Kubernetes Substrate, or up into the guardrails in Security & Policy Enforcement.