cert-manager

cert-manager is the controller that makes TLS certificates a Kubernetes object instead of a chore somebody has to remember. You declare a Certificate — this hostname, from this issuer, stored in this Secret — and cert-manager takes it from there: it requests the certificate from a real certificate authority, writes the result into a kubernetes.io/tls Secret your Ingress or workload already knows how to read, and then watches the clock, renewing automatically well before expiry, forever, without a human touching it again. It talks to the free, automated Let's Encrypt certificate authority over the ACME protocol out of the box, and just as happily to a private CA, HashiCorp Vault's PKI engine, or Venafi — the same declarative object, the same renewal guarantee, whichever authority is actually signing. It's the answer to a genuinely boring, genuinely dangerous problem: certificates expire on a fixed clock that doesn't care whether anyone was watching, and an outage caused by a forgotten renewal is one of the most avoidable a platform team can have.

What cert-manager is, and the problem it solves

☺ Like you're 10: It's the robot that renews your website's ID card automatically, weeks before the old one expires, so nobody ever finds out the hard way.

cert-manager started life at Jetstack and is now developed as its own CNCF project, installed the way most cluster add-ons are — a Helm chart, run once per cluster: helm repo add jetstack https://charts.jetstack.io && helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --set crds.enabled=true. That one install adds a handful of custom resources to the API server — Issuer, ClusterIssuer, Certificate, CertificateRequest, Order, and Challenge — and a controller that watches all of them, following exactly the same declarative-API, reconcile-the-gap pattern every built-in Kubernetes controller uses. That's not incidental framing: cert-manager is the textbook example of "adopt an existing operator instead of writing your own," the case Operators & CRDs reaches for when it asks whether a problem is really yours to solve from scratch.

The problem underneath is almost insultingly simple to state and expensive to get wrong: a TLS certificate has a hard expiry date, HTTPS traffic to an expired one gets a browser warning or an outright connection failure, and "someone will renew it manually before then" is a promise that survives right up until that someone is on vacation, has changed teams, or the reminder calendar invite got silently declined eight months ago. Doing this by hand — buy a cert, download it, base64 it into a Secret, redeploy, repeat every 90 days or every year, across every domain the platform serves — doesn't fail often, but it fails at the worst possible time, and it fails silently until the exact second it becomes very loud.

Issuer and ClusterIssuer: where certificates actually come from

☺ Like you're 10: An Issuer names which "front desk" hands out certificates — one just for your team's rooms, or one every room in the building can use.

Neither Issuer nor Certificate does anything on its own — a Certificate always names an issuer via issuerRef, and that issuer is the thing that actually knows how to talk to a real certificate authority. Issuer is namespaced: it can only be referenced by Certificates in the same namespace, which is exactly the property you want when different teams should be limited to different CAs, or when a namespace-scoped credential (a DNS provider API token, say) shouldn't be readable outside that namespace. ClusterIssuer is the same object cluster-scoped instead — one letsencrypt-prod that every namespace in the cluster can reference — which is by far the more common shape for a single shared public CA, and the one nearly every getting-started guide uses first.

A public-facing production issuer is almost always the ACME type, pointed at Let's Encrypt's production directory:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: platform-team@acme.io          # expiry + abuse notices go here
    privateKeySecretRef:
      name: letsencrypt-prod-account-key   # cert-manager creates + owns this
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx

The ACME dance: how an automated certificate actually gets issued

☺ Like you're 10: Proving you own a website means proving you can change something on it — either a special file, or a special DNS record — and the robot proves it for you.

ACME (Automatic Certificate Management Environment, RFC 8555) is the protocol Let's Encrypt runs, and it works by proving domain control programmatically rather than trusting a human's word for it. A Certificate creates a CertificateRequest, which creates an Order with the CA, which creates one Challenge per domain name that needs proving. Each Challenge gets solved one of two ways: HTTP-01 serves a token at a well-known URL path that the CA fetches over plain HTTP, and DNS-01 publishes a token as a TXT record that the CA looks up over DNS. Once every Challenge is satisfied, the CA issues the certificate, cert-manager writes it into the Secret named on the Certificate, and the whole chain of intermediate objects (CertificateRequest, Order, Challenge) gets cleaned up automatically, leaving just the Certificate and its Secret behind.

The two solver types trade off in ways worth knowing before you pick one:

DNS-01 also has a real advantage HTTP-01 can't match: it needs no public-facing endpoint at all, so it's the only practical route to a browser-trusted certificate for something that never faces the internet. A wildcard issuer against Cloudflare's DNS API looks like this:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: platform-team@acme.io
    privateKeySecretRef:
      name: letsencrypt-dns-account-key
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token   # scoped to DNS-edit on one zone only
              key: api-token
        selector:
          dnsZones: ["acme.io"]

One more nuance worth knowing before it surprises you: a Certificate's duration field is a request, not a guarantee. A ca or vault Issuer will honor it faithfully, but Let's Encrypt ignores it outright and always issues a fixed 90-day certificate regardless of what you asked for — the field only takes effect against issuers that actually let the requester choose.

The Certificate resource, and the ingress-shim shortcut

☺ Like you're 10: You can either write out the certificate request yourself, or just put a sticky note on your Ingress and let the robot write the request for you.

The object you actually own in Git is the Certificate — everything in the previous section (CertificateRequest, Order, Challenge) is machinery cert-manager creates and deletes on your behalf. Writing it explicitly, next to the Ingress that will consume its Secret, keeps the relationship visible and reviewable in a pull request:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: shop-tls
  namespace: shop
spec:
  secretName: shop-tls-secret        # the Ingress's spec.tls.secretName below
  duration: 2160h                    # 90d — ignored by Let's Encrypt, honored by ca/vault
  renewBefore: 720h                  # renew 30d before expiry; default is ~2/3 of lifetime
  dnsNames:
    - shop.example.com
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-ingress
  namespace: shop
spec:
  ingressClassName: nginx
  tls:
    - hosts: [shop.example.com]
      secretName: shop-tls-secret    # cert-manager keeps this Secret current
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service: { name: checkout, port: { number: 80 } }

cert-manager also ships ingress-shim, a controller baked into the same deployment that watches every Ingress for a cert-manager.io/cluster-issuer (or cert-manager.io/issuer) annotation and auto-generates the identical Certificate object for you — read the spec.tls block, invent a matching Certificate, done:

metadata:
  name: shop-ingress
  namespace: shop
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod   # ingress-shim does the rest
spec:
  ingressClassName: nginx
  tls:
    - hosts: [shop.example.com]
      secretName: shop-tls-secret
  rules: [ ... ]                     # same rules as above

Since cert-manager 1.15, the same shim pattern extends to the Gateway API — a cert-manager.io/cluster-issuer annotation on a Gateway's TLS-terminating listener does the same auto-generation, so the shortcut isn't Ingress-only anymore.

Day-to-day commands

☺ Like you're 10: A handful of commands answer the only three questions that matter: is it ready, why isn't it ready, and when does it expire?

The Order and Challenge objects only exist mid-issuance, which makes them the single best place to look when a certificate is stuck — kubectl describe challenge is where the actual ACME error text (a blocked port, an unresolvable DNS record, a wrong API token) shows up in full.

# --- state and health ---
$ kubectl get certificate -A                          # READY column: True/False for every cert
$ kubectl describe certificate shop-tls -n shop        # conditions + events, start here
$ kubectl get certificaterequest,order,challenge -n shop   # only present mid-issuance
$ kubectl describe challenge -n shop $(kubectl get challenge -n shop -o name)  # the real error text

# --- cmctl, cert-manager's own CLI (replaces the older kubectl-cert_manager plugin) ---
$ cmctl status certificate shop-tls -n shop            # human-readable full picture, one command
$ cmctl renew shop-tls -n shop                          # force a renewal right now
$ cmctl check api                                       # is the webhook/API actually healthy

# --- reading the cert cert-manager actually wrote ---
$ kubectl get secret shop-tls-secret -n shop -o jsonpath='{.data.tls\.crt}' \
  | base64 -d | openssl x509 -noout -dates -subject -issuer

# --- installing/upgrading the controller itself ---
$ helm repo add jetstack https://charts.jetstack.io && helm repo update
$ helm upgrade --install cert-manager jetstack/cert-manager \
    --namespace cert-manager --create-namespace --set crds.enabled=true

Gotchas and failure modes

☺ Like you're 10: Almost every real cert-manager incident is one of four things: too many requests, a blocked doorway, the wrong team having a master key, or nobody watching the clock.

A CDN or WAF in front of the cluster can silently break HTTP-01. The solver needs the ACME CA to reach /.well-known/acme-challenge/<token> on port 80 directly against the origin — but Cloudflare's orange-cloud proxying, a caching layer, or a WAF rule blocking unrecognized paths can all intercept that request before it reaches the cluster, and the failure shows up as a generic "unauthorized" or timeout on the Challenge, not an obvious "your CDN is in the way" message. Switching that one Issuer to DNS-01 sidesteps the problem entirely, since it never needs an inbound HTTP request at all.

A ClusterIssuer is cluster-wide, and that's a real RBAC question, not a footnote. Any namespace can reference a shared ClusterIssuer by name, which is convenient right up until a hostile or careless workload in one namespace requests a certificate for a hostname it has no business claiming. If different teams need different trust boundaries — one namespace gets your public CA, another gets a restricted internal one — reach for namespaced Issuer objects instead, and gate who can create Certificate resources at all through RBAC & admission control.

Renewal failures are silent until the certificate is actually gone. A DNS provider API token that got rotated, a Vault mount that got unsealed differently, a solver misconfiguration introduced last sprint — none of these break the currently-valid certificate. They just quietly fail every renewal attempt in the background, and nothing user-facing changes until the old certificate finally hits its hard expiry, at which point it changes everywhere at once. Alert on the Certificate's Ready condition going False, and on cert-manager's own certmanager_certificate_expiration_timestamp_seconds metric crossing a threshold well before the actual expiry date — the kind of proactive signal Observability on Kubernetes is built around, rather than the reactive kind you get from users reporting a browser warning.

Rate limits are per domain, and staging exists to protect them. Every failed attempt against Let's Encrypt's production API still counts against that domain's weekly limit, so a broken solver left retrying in a loop overnight can exhaust a real budget before a human ever sees the alert. Test new issuers, new solvers, and new DNS credentials against staging first, every time — it's the single cheapest insurance available here.

cert-manager vs. the alternatives

☺ Like you're 10: You could do this by hand, buy it from your cloud provider, or let a service mesh handle its own internal certs — cert-manager is usually the one piece that ties all three together.

cert-manager isn't the only way to get TLS onto a cluster, and knowing what the alternatives actually trade away makes it obvious when to reach for it and when not to bother.

These aren't strictly exclusive: a cluster running a service mesh for internal mTLS still typically runs cert-manager for the public edge, and cert-manager's own vault issuer means it's often the front door to a PKI that's really being operated by Vault underneath. Platform Engineering's own cert-manager tool page goes further into the CertificateRequest-approval workflow, trust-manager (the sibling project that distributes trusted CA bundles cluster-wide rather than issuing leaf certs), and multi-tenant issuer design — worth reading next if you're operating this at platform scale rather than pointing one cluster at Let's Encrypt. For the broader question of what "we have TLS" is actually supposed to defend against, DevSecOps's cryptography & key management page covers the threat-model side of the same renewal loop described here.