Istio

Istio is a CNCF-graduated service mesh: it puts a programmable proxy in front of every workload so that encryption, identity, authorization, retries, timeouts, circuit breaking, traffic splitting and golden-signal telemetry stop being things each application has to implement and become things the platform provides by configuration. It solves the problem that shows up the moment you have more than a handful of services — the same dozen cross-cutting concerns re-implemented badly in every language, with no consistent way to answer “is this call encrypted, who is allowed to make it, and why is it slow?”

What Istio is and the problem it solves

☺ Like you’re 10: Every program needs the same boring safety gear. Istio hands out the gear so nobody has to knit their own.

Take any real microservice estate and list what each service must do beyond its actual business logic: encrypt outbound calls and verify the other side; retry a call that failed for a silly reason but not one that failed for a real reason; give up after a sensible timeout; stop hammering a dependency that is already on fire; emit request rate, error rate and latency in a format your dashboards understand; propagate trace headers. That list is identical for every service, and yet it is normally solved by a library — which means it is solved once per language, upgraded never, and configured inconsistently.

Moving the concerns out of the application

A service mesh moves that list out of the process and into a proxy that sits on the network path of every request. Because the proxy sees every byte in and out, it can encrypt, authorize, retry, and measure without a single line of application change — and because the proxies are configured centrally, the behaviour is uniform across a fleet written in six languages. Istio is the most feature-complete implementation of that idea: it uses Envoy as its proxy and exposes a large, expressive configuration surface on top.

Graduated, Envoy-based, and deliberately large

Istio graduated in the CNCF in 2023 and is on the official CNPE tool list. It is honest about being a big system: it has more knobs than Linkerd and a correspondingly steeper curve. That size is the trade — you get L7 routing rules, per-subset load-balancing policy, outlier detection, JWT validation, multi-cluster identity and a full telemetry pipeline, and in exchange you take on a distributed system that your platform team now operates.

☺ Like you’re 10: Istio is the big Swiss-army knife. Brilliant if you need the corkscrew and the tiny scissors; heavy in your pocket if all you wanted was the blade.

Where Istio fits in a platform

☺ Like you’re 10: Istio isn’t the road — it’s the traffic police, the seatbelts and the dashcams that ride on top of the road.

Istio is a data-plane capability with its own small control plane, sitting in the connectivity and security layer of the platform described in Platform Architecture. It is emphatically not a CNI: it does not give pods IP addresses or move packets between nodes. It assumes the flat pod network from Networking & Service Connectivity already works, and layers L4/L7 behaviour on top of it.

Its neighbours, and who does what

Confusion about the mesh is almost always confusion about layering. The CNI (Calico, Cilium, Flannel) delivers packets and enforces NetworkPolicy at L3/L4 by IP and label. The mesh enforces policy at L4/L7 by cryptographic identity and by HTTP verb and path. They are complementary, and a mature platform runs both: NetworkPolicy as the coarse blast-radius fence, AuthorizationPolicy as the fine-grained, identity-aware rule. Above them, the ingress layer decides what gets in from outside — and here Istio can play both roles, acting as your north-south gateway as well as your east-west mesh.

Progressive delivery rides on the mesh

Because Istio can split traffic by weight independently of pod counts, it is the routing engine behind mesh-driven canaries. Argo Rollouts and Flagger both drive Istio: they write the weights into a VirtualService for you, watch metrics, and either advance or abort. This is exactly the separation of deploy from release that CI/CD & Progressive Delivery is built on, and the Istio manifests below are what those controllers are mutating under the hood.

CNPE domain relevance

Istio lands hardest in the Security & Policy domain (workload identity, mTLS, authorization), meaningfully in GitOps & Continuous Delivery (weighted routing for canaries), and touches Observability (uniform golden-signal metrics and traces without app instrumentation) and Platform Architecture (multi-cluster east-west connectivity, covered in Multi-Cluster). All of its configuration is plain Kubernetes custom resources, so it is delivered by the same reconciler as everything else — see GitOps Workflows.

How Istio works

☺ Like you’re 10: One office in the middle (istiod) tells all the little messengers what to do, and hands each one an ID badge that can’t be faked.

Istio has exactly two halves. The control plane is a single binary called istiod. The data plane is the set of proxies that actually carry traffic — either an Envoy sidecar per pod, or, in ambient mode, a per-node ztunnel plus optional per-namespace waypoints.

istiod — the control plane

Modern Istio collapsed what used to be several components (Pilot, Citadel, Galley) into one deployment in the istio-system namespace. istiod does four things. It watches the Kubernetes API for Services, EndpointSlices, Pods and Istio custom resources. It translates that state into Envoy configuration and pushes it to every proxy over the xDS protocol (clusters, listeners, routes, endpoints, secrets). It acts as the mesh certificate authority, issuing each workload a short-lived X.509 certificate whose SAN is a SPIFFE identity of the form spiffe://cluster.local/ns/<namespace>/sa/<serviceaccount> — derived from the pod’s ServiceAccount token, so identity is minted from something Kubernetes already vouches for. And it runs the admission webhooks that validate your CRs and inject sidecars.

The sidecar data plane and injection

In the classic model, an istio-proxy container (Envoy) is injected into each pod and an init container programs iptables rules so that all inbound traffic is redirected to port 15006 and all outbound to 15001. The application is untouched and unaware. Injection is opt-in per namespace via a label — either the simple istio-injection=enabled or, on a revisioned install, istio.io/rev=<revision>, which is what lets you run two control-plane versions side by side and migrate namespace by namespace.

☺ Like you’re 10: You don’t add the messenger yourself — you put a sticker on the room, and every new desk in that room gets a messenger automatically.

Ambient mode — the lighter data plane

Ambient splits the mesh by layer instead of by pod. A per-node DaemonSet called ztunnel handles L4 only: mTLS, identity and basic telemetry for every enrolled pod on that node, tunnelling traffic over HBONE (HTTP/2 CONNECT over mTLS). If — and only if — a namespace or service needs L7 features (HTTP routing rules, header-based matching, AuthorizationPolicy on methods and paths), you add a waypoint proxy, a normal Envoy deployment that traffic for that namespace is routed through. You enrol a namespace by labelling it istio.io/dataplane-mode=ambient. The payoff is real: no per-pod proxy, so no per-pod memory tax, no pod restarts to upgrade the mesh, and no sidecar lifecycle problems at all. You pay for L7 only where you use it.

The resources Istio introduces

Istio adds two families of CRDs. Under networking.istio.io: VirtualService (routing rules — match on host, header, path, then route, split, retry, timeout, fault-inject), DestinationRule (what happens after routing — named subsets, load-balancer choice, connection pools, outlier detection, TLS mode), Gateway (a listener at the edge of the mesh, bound to an ingress or egress gateway deployment), ServiceEntry (teach the mesh about a service outside the cluster so you can apply policy to it), and Sidecar (limit which services a proxy is told about). Under security.istio.io: PeerAuthentication, AuthorizationPolicy and RequestAuthentication. A Telemetry resource under telemetry.istio.io tunes access logs, metrics and tracing.

The resources you will actually write

☺ Like you’re 10: Three little files do most of the work: one that splits traffic, one that says “everyone must whisper in code,” and one that says exactly who may knock on which door.

These are the shapes worth having in your fingers. Note that since Istio 1.22 the stable API version for the core resources is v1 in networking.istio.io, security.istio.io and telemetry.istio.io alike (you will still see networking.istio.io/v1alpha3 and v1beta1 in older examples; the schemas are compatible, and EnvoyFilter deliberately stays at v1alpha3).

Traffic management: a weighted canary

The pair you almost always write together. The DestinationRule defines subsets — named groups of pods selected by label — and the VirtualService sends a percentage of requests to each. Weights within a route must sum to 100.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: checkout
  namespace: prod
spec:
  host: checkout.prod.svc.cluster.local   # the Kubernetes Service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST               # ROUND_ROBIN | RANDOM | LEAST_REQUEST
    connectionPool:
      tcp:  { maxConnections: 100 }
      http: { http2MaxRequests: 1000, maxRequestsPerConnection: 10 }
    outlierDetection:                     # this IS Istio's circuit breaker
      consecutive5xxErrors: 5             # 5 bad responses in a row...
      interval: 10s                       # ...checked every 10s...
      baseEjectionTime: 30s               # ...ejects that endpoint for 30s
      maxEjectionPercent: 50              # never eject more than half the pool
  subsets:
    - name: v1
      labels: { version: v1 }             # matches pod labels, not Service labels
    - name: v2
      labels: { version: v2 }
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: checkout
  namespace: prod
spec:
  hosts:
    - checkout                            # short name resolves in this namespace
  http:
    - match:                              # first match wins — order matters
        - headers:
            x-canary: { exact: "true" }
      route:
        - destination: { host: checkout, subset: v2 }
    - route:                              # default route: the 90/10 split
        - destination: { host: checkout, subset: v1 }
          weight: 90
        - destination: { host: checkout, subset: v2 }
          weight: 10                      # Rollouts/Flagger rewrite this number
      timeout: 3s
      retries:
        attempts: 3
        perTryTimeout: 1s
        retryOn: 5xx,reset,connect-failure

☺ Like you’re 10: The DestinationRule names the teams; the VirtualService decides how many visitors each team gets.

Security: strict mTLS and a default-deny authorization posture

Two resources take a namespace from “anything can call anything in plaintext” to zero-trust. PeerAuthentication with mode: STRICT requires that every inbound connection present a mesh certificate. AuthorizationPolicy then decides which identities may do what — and a policy whose whole spec is the empty map {} is the deny-all idiom you must know cold.

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: prod              # in istio-system with no selector = mesh-wide
spec:
  mtls:
    mode: STRICT               # STRICT | PERMISSIVE | DISABLE | UNSET
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: deny-all
  namespace: prod
spec:
  {}                           # no selector, no rules, no action field
                               # => ALLOW with zero rules = deny everything
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: checkout-allow-frontend
  namespace: prod
spec:
  selector:
    matchLabels: { app: checkout }
  action: ALLOW
  rules:
    - from:
        - source:
            principals:        # SPIFFE identity, NOT an IP
              - cluster.local/ns/prod/sa/frontend
      to:
        - operation:
            methods: ["GET", "POST"]
            paths: ["/api/cart*"]
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: block-admin-path
  namespace: prod
spec:
  action: DENY                 # DENY is evaluated BEFORE any ALLOW
  rules:
    - to:
        - operation:
            paths: ["/admin*"]

Two evaluation rules matter more than any field. First, DENY policies are evaluated before ALLOW policies — a matching DENY wins outright. Second, if any ALLOW policy selects a workload, that workload is default-deny for everything not explicitly allowed. That is why the empty-spec policy above locks a namespace down; there is also the explicit form action: DENY with rules: [{}], which matches every request. action: AUDIT is the safe third option: it logs what would have been denied without denying it, and is the correct way to roll a policy out.

The front door: Gateway, and the Gateway API

Istio’s own Gateway binds a listener on a gateway deployment; a VirtualService then attaches routes to it. But the strategic direction — stated by the project itself — is the vendor-neutral Kubernetes Gateway API, which Istio implements as a first-class ingress and, via waypoints, for mesh traffic too. New platforms should prefer it.

# --- Istio's own Gateway + VirtualService (the classic ingress pattern) ---
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: web
  namespace: prod
spec:
  selector:
    istio: ingressgateway       # labels of the gateway Deployment's pods
  servers:
    - port: { number: 443, name: https, protocol: HTTPS }
      hosts: ["shop.example.com"]
      tls:
        mode: SIMPLE
        credentialName: shop-tls   # a Secret in the gateway's namespace
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: web
  namespace: prod
spec:
  hosts: ["shop.example.com"]
  gateways: ["web"]             # without this it only applies to mesh traffic
  http:
    - route:
        - destination: { host: frontend, port: { number: 8080 } }
---
# --- The same thing in the Gateway API (the strategic direction) ---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: web
  namespace: prod
spec:
  gatewayClassName: istio
  listeners:
    - name: https
      port: 443
      protocol: HTTPS
      tls:
        mode: Terminate
        certificateRefs:
          - name: shop-tls
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: web
  namespace: prod
spec:
  parentRefs: [{ name: web }]
  hostnames: ["shop.example.com"]
  rules:
    - backendRefs:
        - name: frontend
          port: 8080
---
# --- Reach an external API through the mesh, with policy applied ---
apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
  name: stripe
  namespace: prod
spec:
  hosts: ["api.stripe.com"]
  ports:
    - { number: 443, name: https, protocol: HTTPS }
  resolution: DNS
  location: MESH_EXTERNAL

Day-to-day commands

☺ Like you’re 10: Four commands cover almost everything: check before you install, check your YAML makes sense, check every messenger got the memo, and ask one messenger what it actually believes.

istioctl is the tool. Note that istioctl analyze in particular is a genuinely excellent linter — it catches missing subsets, conflicting policies and unreferenced gateways before they become a 503.

Install and validate

# Will this cluster support Istio? Run BEFORE installing.
istioctl x precheck

# Install with a built-in profile (demo | default | minimal | ambient)
istioctl install --set profile=default -y
istioctl install --set profile=ambient -y        # ztunnel + CNI node agent

# Enrol a namespace — pick ONE of these three, they are mutually exclusive.
# Sidecar mode, default revision:
kubectl label namespace prod istio-injection=enabled
# Sidecar mode on a revisioned install; the value is the revision name you
# chose at install time. istio-injection=enabled wins if both labels are
# present, so remove it first: kubectl label namespace prod istio-injection-
kubectl label namespace prod istio.io/rev=<revision>
# Ambient mode:
kubectl label namespace prod istio.io/dataplane-mode=ambient

# Existing pods are NOT retrofitted — they must be restarted
kubectl rollout restart deployment -n prod

# Lint the mesh config in a namespace (or a file, before you commit it)
istioctl analyze -n prod
istioctl analyze virtualservice.yaml destinationrule.yaml

# Check the install itself: control-plane pods, and which versions are in play
kubectl -n istio-system get pods
istioctl version                 # client, istiod, and every data-plane proxy

Is the config actually where it should be?

# The single most useful command: has every proxy got the latest config?
istioctl proxy-status                 # alias: istioctl ps
# CDS/LDS/EDS/RDS columns should all read SYNCED.
# STALE = the push is in flight or the proxy is wedged.
# NOT SENT = istiod has nothing to send for that resource type.

# What does ONE proxy actually believe? (this is the debugging workhorse)
istioctl proxy-config cluster  checkout-7d9f-abcde.prod
istioctl proxy-config route    checkout-7d9f-abcde.prod --name 8080 -o json
istioctl proxy-config listener checkout-7d9f-abcde.prod --port 15006
istioctl proxy-config endpoint checkout-7d9f-abcde.prod --cluster \
  "outbound|8080|v2|checkout.prod.svc.cluster.local"
istioctl proxy-config secret   checkout-7d9f-abcde.prod   # is the cert there?

# Plain-English summary of one pod's mesh state: mTLS mode, policies, routes
istioctl x describe pod checkout-7d9f-abcde.prod

# Turn up Envoy logging on one proxy while you reproduce a failure
istioctl proxy-config log checkout-7d9f-abcde.prod --level router:debug

# Read the sidecar's own logs — 503s and RBAC denials show up here
kubectl logs checkout-7d9f-abcde -n prod -c istio-proxy

# Ambient: create and inspect a waypoint for a namespace
istioctl waypoint apply -n prod --enroll-namespace
istioctl waypoint list -n prod

When a request fails, the diagnostic order is almost always the same: proxy-status (did the config land?), then proxy-config route and cluster (is my routing rule even present?), then proxy-config endpoint (are there healthy backends behind it?), then the istio-proxy logs (what response flag did Envoy record?). Triage: Networking walks the same ladder from the packet up.

Gotchas and failure modes

☺ Like you’re 10: Most Istio pain isn’t Istio being wrong — it’s something outside the mesh that suddenly isn’t allowed to talk anymore.

STRICT mTLS cuts off everything unmeshed

The classic outage. You apply mtls.mode: STRICT and immediately break every client that isn’t in the mesh: a monitoring scraper outside the namespace, a legacy VM, a CronJob in a namespace you forgot to label, a cloud load balancer health check hitting a pod directly. The correct sequence is always: install the mesh in PERMISSIVE (accept both mTLS and plaintext), enrol and restart all workloads, confirm with istioctl x describe pod that traffic is actually mTLS, and only then flip to STRICT — namespace by namespace, never mesh-wide first.

Sidecar lifecycle races

Historically the ugliest class of bug. If the app container starts before istio-proxy is ready, its first outbound calls fail — a migration job that runs at boot dies mysteriously. And in the other direction, a Job or CronJob pod never reaches Completed, because the app exits but the sidecar keeps running forever. The old workarounds were holdApplicationUntilProxyStarts and having the app curl the proxy’s /quitquitquit endpoint on exit. The real fix is Kubernetes native sidecars — entries in initContainers carrying restartPolicy: Always, which went beta and on by default in Kubernetes 1.29 and reached GA in 1.33 — and which Istio uses to guarantee start-before and stop-after ordering. Ambient mode sidesteps the whole category, because there is no sidecar.

The cost of a proxy per pod — and of telling it everything

Every sidecar costs memory and CPU and adds a hop of latency in each direction. At a few hundred pods that is real money and real p99. Worse, by default istiod tells every proxy about every service in the mesh, so each proxy’s config grows with the size of the mesh — total config volume across the fleet therefore grows roughly with the square of your estate, and pushes get slow. The two levers are the Sidecar resource (restrict a namespace’s proxies to the hosts they actually call) and exportTo/discoverySelectors to scope visibility. If none of that appeals, that is precisely the argument for ambient mode or for Linkerd.

apiVersion: networking.istio.io/v1
kind: Sidecar
metadata:
  name: default
  namespace: prod          # applies to every proxy in this namespace
spec:
  egress:
    - hosts:
        - "./*"            # everything in my own namespace
        - "istio-system/*" # plus the control plane
        - "payments/*"     # plus the one namespace I genuinely call
  outboundTrafficPolicy:
    mode: REGISTRY_ONLY    # block egress to anything without a ServiceEntry

☺ Like you’re 10: Don’t hand every messenger the whole school phone book. Give them the three numbers they actually dial.

Alternatives, and when to choose Istio

☺ Like you’re 10: There is more than one way to give everyone a messenger — and sometimes you don’t need messengers at all.

The honest first question is whether you need a mesh. For a dozen services with simple needs, NetworkPolicy plus a decent HTTP client library plus cert-manager may be enough, and you will have saved yourself a distributed system. Adopt a mesh when you genuinely need mTLS everywhere, uniform L7 reliability, identity-based authorization, or traffic shifting across many services at once.

Sidecar or ambient?

Exam relevance and going further

☺ Like you’re 10: On exam day the Istio website is locked. Whatever you can type from memory is what you have.

Istio is on the official CNPE tool list and sits squarely in the Security & Policy domain, with a supporting role in the delivery and observability domains. Expect to be asked to enforce mTLS on a namespace, express “only this ServiceAccount may call this workload,” or shift a percentage of traffic to a new version.

⚖ CNPA vs CNPE — That narrow allowlist is a CNPE mechanic: CNPE is hands-on, so a sliver of official documentation stays reachable. CNPA is stricter still — a fully closed-book multiple-choice exam with zero lookups of any kind, not even kubernetes.io. Knowing Istio’s concepts (mTLS, authorization policy evaluation order, traffic splitting) cold is still worthwhile prep for CNPA’s closed-book recall, even though CNPA won’t have you write a manifest.

What to be able to do cold

• Label a namespace for injection (istio-injection=enabled or istio.io/rev=) and remember that existing pods must be restarted.
• Write a PeerAuthentication with mtls.mode: STRICT, and say where it must live to be mesh-wide (istio-system, no selector).
• Write the deny-all AuthorizationPolicy, then an ALLOW policy using principals of the form cluster.local/ns/<ns>/sa/<sa> with to.operation.methods and paths.
• State the two evaluation rules: DENY is checked before ALLOW; any selecting ALLOW makes the workload default-deny.
• Write a DestinationRule with subsets plus a VirtualService with weights summing to 100 — and know that a subset referenced but not defined is a 503.
• Name the components: istiod, Envoy sidecar, ztunnel (L4), waypoint (L7) — and the difference between sidecar and ambient enrolment.
• Drive istioctl: x precheck, install, analyze, proxy-status, and proxy-config cluster|route|listener|endpoint.

Where to go next on this site, and officially

The lesson this page supports is Networking & Service Connectivity; the policy half belongs to Security & Policy Enforcement; the canaries it powers are in CI/CD & Progressive Delivery with Flagger as the mesh-native driver; the metrics it emits feed Observability; the lighter alternative is Linkerd; the whole shed is mapped on The Tool Landscape. Terms like SPIFFE, sidecar and circuit breaking are defined in the Glossary; the manifests to memorise are on Know Cold; and when traffic misbehaves, start at Triage: Networking.

Officially, and for study only — never during the exam: project documentation at istio.io/latest/docs (start with Concepts → Traffic Management and Security), the ambient guide at istio.io/latest/docs/ambient, source at github.com/istio/istio, the proxy at envoyproxy.io, the identity spec at spiffe.io, the strategic routing API at gateway-api.sigs.k8s.io, and the ecosystem map at landscape.cncf.io.