KEDA

KEDA — Kubernetes Event-Driven Autoscaling — is a CNCF-graduated add-on that lets a workload’s replica count follow something outside the cluster: the depth of an SQS queue, the consumer lag on a Kafka topic, the length of a Redis list, the answer to a PromQL query, or simply the time of day. Its headline trick is the one the built-in Horizontal Pod Autoscaler cannot do — scaling all the way to zero and back up again when a single message arrives. It solves the platform problem of “our workers burn money idling all night, and CPU percentage tells us nothing about how far behind we are.”

What KEDA is and the problem it solves

☺ Like you’re 10: It counts the pile of work waiting outside the cluster and opens exactly as many desks as the pile needs — including none.

KEDA started as a joint Microsoft/Red Hat project in 2019 and graduated in the CNCF in August 2023. It is a small, single-purpose add-on: a couple of Deployments and a handful of CRDs. It does not fork Kubernetes autoscaling, replace the HPA, or ask you to rewrite your application. You install it once, and thereafter any team can attach a ScaledObject to a Deployment and get event-driven scaling declaratively — exactly the shape a platform team wants from a self-service capability.

Why CPU is the wrong signal for a worker

The stock HPA scales on resource utilisation, and for a request-serving web tier that is often a fine proxy: more traffic, more CPU, more pods. For an event consumer it is close to useless. A worker that pulls a message, calls three APIs and waits on I/O may sit at 8% CPU while a hundred thousand messages pile up behind it. CPU says “relax”; the business says “we are two hours behind.” The signal you actually care about lives outside the cluster — in the broker, the queue, the topic, the database — and Kubernetes has no native way to see it.

The second failure is the floor. On any default cluster an HPA’s minReplicas must be at least 1, because its algorithm scales the current replica count by the metric ratio and zero collapses the arithmetic. (Kubernetes has carried a long-standing alpha feature gate, HPAScaleToZero, that permits minReplicas: 0 for object and external metrics, but it is off by default and KEDA does not depend on it.) Every idle worker pool therefore pays for at least one pod, per environment, forever — multiply that by forty microservices across dev, staging and prod and the FinOps conversation writes itself.

What KEDA adds, in one sentence

KEDA supplies two things Kubernetes lacks: a library of adapters (“scalers”) that read external systems and translate them into Kubernetes external metrics, and an activation path between 0 and 1 replicas that the HPA cannot express. Everything from 1 to N is still ordinary HPA behaviour — the stabilisation windows, the scaling policies, the tolerance — because KEDA generates a real HPA and lets it do that job.

Where it fits in a platform, and its CNPE relevance

☺ Like you’re 10: KEDA is a small part in the platform’s engine room. It doesn’t build or deploy anything — it decides how many copies of somebody else’s thing should be running.

In the reference architecture, KEDA belongs to the workload/runtime plane, sitting beside the HPA, VPA and the cluster-level autoscalers. The platform team installs and operates it as a shared capability; tenants consume it purely through YAML in their own namespaces, delivered like everything else through GitOps. That division is the point: nobody needs cluster-admin, broker credentials never leave a Secret, and the scaling policy for a service lives in the same repo as the service.

Its neighbours, and who does what

KEDA is easy to confuse with its neighbours. The HPA changes replica count from resource or custom metrics; KEDA drives one of these with external signals. The VPA changes the size of each pod, not the number — a different axis entirely. Karpenter and the Cluster Autoscaler add and remove nodes once KEDA’s new pods are Pending; KEDA never provisions machines, so a ScaledObject asking for 200 replicas on a cluster with no headroom simply produces 200 Pending pods. Prometheus is a common trigger source rather than a competitor, and OpenCost is how you prove afterwards that scale-to-zero saved money. Knative Serving overlaps most directly — the same trick for synchronous HTTP traffic — and is covered in the comparison below.

Where it shows up on the CNPE blueprint

KEDA is not on the official CNPE tool list, so no task will hand you a broken ScaledObject and ask you to fix it by name. It is on this site anyway because event-driven and scale-to-zero autoscaling is core platform work, and because it is the cleanest worked example of two things the exam does care about: how Kubernetes autoscaling actually composes, and how an operator extends the platform through custom resources rather than a bespoke API. Study it as understanding, not as memorised YAML — the lesson that matters for the exam is Scaling & Scheduling.

How it works — architecture, components, CRDs

☺ Like you’re 10: Three little programs: one watches the queue and turns the light on, one answers “how big is the pile?” when Kubernetes asks, and one checks your YAML for silly mistakes.

A KEDA install (conventionally in a namespace called keda) is three Deployments. Knowing which one is misbehaving is most of KEDA troubleshooting.

The three components

The operator (keda-operator) is the controller. It watches ScaledObject and ScaledJob resources, instantiates the scalers they declare, and runs its own polling loop at pollingInterval (default 30 seconds). Its exclusive job is the zero boundary: when a workload sits at zero and the operator sees activity, it scales the target to one directly, without involving the HPA; when the source has been quiet for cooldownPeriod (default 300 seconds), it scales back to zero. It also creates, updates and garbage-collects the generated HPA.

The metrics adapter (keda-operator-metrics-apiserver) is a Kubernetes external metrics API server. It registers an APIService for external.metrics.k8s.io/v1beta1, and when the HPA controller asks “what is the current value of this external metric?” — roughly every 15 seconds, the kube-controller-manager’s HPA sync period — the adapter runs the relevant scaler and answers. Note the consequence: once a workload is above zero, the HPA sync period, not pollingInterval, governs how fast you react.

The admission webhook (keda-admission-webhooks) validates resources at write time. Its most valuable rule rejects a ScaledObject whose scaleTargetRef points at a workload already managed by another HPA — the single most common self-inflicted KEDA outage, caught at kubectl apply instead of at 3am.

The custom resources

Scalers — the adapter library

A scaler knows how to ask one external system for a number, and whether that number counts as “active.” KEDA ships more than seventy, and the shape is always the same: a type, a metadata block of connection details, a threshold (the target value per replica), and an activationThreshold (the value above which zero becomes one). If nothing built-in fits, the external and external-push types let you implement KEDA’s gRPC interface yourself, and the metrics-api scaler reads a number out of any JSON endpoint.

When a ScaledObject lists several triggers, each becomes a separate external metric on the generated HPA, and the HPA takes the largest desired replica count across them. That is how you express “scale on queue depth, but never below four replicas during business hours” — a queue trigger plus a cron trigger. Since v2.12, advanced.scalingModifiers lets you combine trigger values with a formula instead of taking the maximum.

The resources you will actually write

☺ Like you’re 10: Three little YAML files: “watch this queue,” “here’s the password,” and “run one job per message.”

A ScaledObject, annotated

This is the resource an application team writes. Read it as three parts: what to scale, between what bounds, and on what signal.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: thumbnailer                 # generated HPA will be keda-hpa-thumbnailer
  namespace: media
  annotations:
    # Maintenance switches. "paused-replicas" pins the workload and stops all
    # scaling; "paused: true" freezes at the CURRENT count. Remove to resume.
    autoscaling.keda.sh/paused-replicas: "3"
spec:
  scaleTargetRef:
    name: thumbnailer               # a Deployment in THIS namespace by default
    # apiVersion: apps/v1           # set kind/apiVersion for StatefulSets or
    # kind: Deployment              # any CR exposing a /scale subresource
  minReplicaCount: 0                # 0 = scale-to-zero. HPA minReplicas becomes 1;
                                    # KEDA itself owns the 0 <-> 1 hop.
  maxReplicaCount: 60
  # idleReplicaCount: 0             # optional, and STRICTLY LESS than minReplicaCount --
                                    # so it only makes sense with a non-zero floor, e.g.
                                    # minReplicaCount: 2 + idleReplicaCount: 0 meaning
                                    # "idle at 0, but floor at 2 once there is work".
                                    # Setting both to 0 is rejected as invalid.
  pollingInterval: 15               # seconds; ONLY governs the operator's own loop --
                                    # i.e. the 0 -> 1 activation and the 1 -> 0
                                    # deactivation. Default 30.
  cooldownPeriod: 300               # seconds of inactivity before 1 -> 0. Default 300.
  fallback:                         # if the scaler ERRORS this many times in a row,
    failureThreshold: 3             # hold at a safe replica count instead of
    replicas: 6                     # collapsing. Requires metricType: AverageValue.
  advanced:
    restoreToOriginalReplicaCount: true   # on ScaledObject delete, put replicas back
    horizontalPodAutoscalerConfig:
      behavior:                     # this is the STOCK HPA behaviour block, passed
        scaleDown:                  # straight through to the generated HPA
          stabilizationWindowSeconds: 300
          policies:
            - type: Percent
              value: 50
              periodSeconds: 60
  triggers:
    - type: aws-sqs-queue
      metricType: AverageValue      # default; the HPA divides by replica count
      metadata:
        queueURL: https://sqs.eu-west-1.amazonaws.com/123456789012/thumbnails
        awsRegion: eu-west-1
        queueLength: "25"           # THRESHOLD: aim for ~25 messages per replica
        activationQueueLength: "1"  # ACTIVATION: >1 message wakes us from zero
      authenticationRef:
        name: sqs-creds             # a TriggerAuthentication in this namespace
    - type: cron                    # second trigger: never fewer than 4 in office hours
      metadata:
        timezone: Europe/London
        start: "0 8 * * 1-5"
        end: "0 19 * * 1-5"
        desiredReplicas: "4"

The two thresholds are the part people get wrong, so say it out loud once: queueLength: "25" is the target ratio the HPA steers toward — 250 messages means 10 replicas — while activationQueueLength: "1" is a separate on/off switch that only matters at zero. Set activation too high and the workload never wakes; leave it at the default of 0 with a noisy source and it never sleeps.

TriggerAuthentication and its cluster-scoped twin

Never put a broker password in trigger.metadata. TriggerAuthentication maps named scaler parameters onto Secret keys, environment variables on the target’s own pod, or a cloud identity — and it composes neatly with External Secrets, so the actual value is synced from a vault and never touches Git. See Secrets Management for the wider pattern.

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: sqs-creds
  namespace: media
spec:
  secretTargetRef:                  # map SCALER PARAMETER -> secret key
    - parameter: awsAccessKeyID     # parameter names are defined by the scaler
      name: sqs-credentials
      key: AWS_ACCESS_KEY_ID
    - parameter: awsSecretAccessKey
      name: sqs-credentials
      key: AWS_SECRET_ACCESS_KEY
---
# Cluster-scoped: the platform team publishes ONE of these; every tenant
# references it with authenticationRef.kind: ClusterTriggerAuthentication.
apiVersion: keda.sh/v1alpha1
kind: ClusterTriggerAuthentication
metadata:
  name: platform-workload-identity
spec:
  podIdentity:
    provider: aws                   # or azure-workload / gcp — no static keys at all
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata: { name: ledger-consumer, namespace: payments }
spec:
  scaleTargetRef: { name: ledger-consumer }
  minReplicaCount: 2                # a hot path: never scale to zero
  maxReplicaCount: 40
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka.data.svc:9092
        consumerGroup: ledger
        topic: ledger-events
        lagThreshold: "500"         # ~500 messages of lag per replica
        activationLagThreshold: "10"
        excludePersistentLag: "true"  # ignore partitions whose lag never moves
      authenticationRef:
        kind: ClusterTriggerAuthentication
        name: platform-workload-identity

ScaledJob — one Job per unit of work

A ScaledObject keeps long-running pods that loop over a queue. A ScaledJob is for work that is naturally one message, one execution, then exit — a video transcode, a nightly report, a machine-learning inference batch. KEDA creates real Kubernetes Jobs and no HPA is involved at all, which means Job semantics apply: backoffLimit, restart policy, TTL, and the guarantee that a pod that finishes is gone rather than idling.

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: transcoder
  namespace: media
spec:
  jobTargetRef:                     # a normal Kubernetes JobSpec
    parallelism: 1
    completions: 1
    backoffLimit: 4
    template:
      spec:
        restartPolicy: Never
        containers:
          - name: transcode
            image: registry.example.com/transcode:2.4.1
            resources:
              requests: { cpu: "1", memory: 2Gi }
  pollingInterval: 20               # how often KEDA checks the queue
  maxReplicaCount: 30               # ceiling on CONCURRENT Jobs
  successfulJobsHistoryLimit: 3     # keep 3 finished Jobs for debugging
  failedJobsHistoryLimit: 5
  scalingStrategy:
    strategy: accurate              # subtract running Jobs from the queue depth
  triggers:
    - type: rabbitmq
      metadata:
        protocol: amqp
        queueName: transcode
        mode: QueueLength
        value: "1"                  # one Job per queued message
        activationValue: "0"
      authenticationRef:
        name: rabbit-creds

Day-to-day commands

☺ Like you’re 10: There’s no special KEDA program to run — it’s all kubectl, and mostly you’re asking “is it awake, and why not?”

Installing and checking the install

helm repo add kedacore https://kedacore.github.io/charts && helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace

kubectl -n keda get deploy      # expect: keda-operator, keda-operator-metrics-apiserver,
                                #         keda-admission-webhooks
kubectl get crd | grep keda.sh  # scaledobjects, scaledjobs, triggerauthentications, ...

# THE health check for the metrics path: is the external metrics API being served?
kubectl get apiservice v1beta1.external.metrics.k8s.io
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1" | head -c 400

Inspecting a ScaledObject

The printer columns are the fastest read in KEDA. READY means the triggers parsed and connected; ACTIVE means the source currently has work, which is the flag that separates “correctly asleep” from “broken.”

kubectl -n media get scaledobject
#  columns abridged below -- the real output also carries AUTHENTICATION, FALLBACK and AGE
#  NAME        SCALETARGETKIND      SCALETARGETNAME  MIN  MAX  TRIGGERS       READY  ACTIVE  PAUSED
#  thumbnailer apps/v1.Deployment   thumbnailer      0    60   aws-sqs-queue  True   False   Unknown

kubectl -n media describe scaledobject thumbnailer   # conditions + events say WHY
kubectl -n media get hpa keda-hpa-thumbnailer -o yaml  # the HPA KEDA generated & owns
kubectl -n media get hpa keda-hpa-thumbnailer         # TARGETS column = current/threshold

# KEDA derives the external metric name per trigger (s0-, s1-, ... plus scaler detail),
# so read it off the generated HPA rather than guessing at it.
kubectl -n media get hpa keda-hpa-thumbnailer \
  -o jsonpath='{.spec.metrics[*].external.metric.name}{"\n"}'

# Then ask the external metrics API exactly what the HPA is being told.
kubectl get --raw \
  "/apis/external.metrics.k8s.io/v1beta1/namespaces/media/<that-metric-name>" | jq

kubectl -n keda logs deploy/keda-operator --tail=100          # scaler + activation errors
kubectl -n keda logs deploy/keda-operator-metrics-apiserver   # metric-serving errors

Pausing, resuming and removing

# Freeze at a fixed count for a migration or an incident (survives a broker outage).
# --overwrite, because kubectl annotate errors if the key is already present.
kubectl -n media annotate --overwrite scaledobject thumbnailer autoscaling.keda.sh/paused-replicas="3"
# Freeze wherever it happens to be right now.
kubectl -n media annotate --overwrite scaledobject thumbnailer autoscaling.keda.sh/paused="true"
# Resume — the trailing hyphen REMOVES the annotation.
kubectl -n media annotate scaledobject thumbnailer autoscaling.keda.sh/paused-replicas-

kubectl -n media delete scaledobject thumbnailer   # also deletes the generated HPA
# ...replicas are left where they are unless advanced.restoreToOriginalReplicaCount: true

kubectl -n media get events --field-selector involvedObject.kind=ScaledObject

Because all of this is kubectl on ordinary objects, the drills in the command reference transfer directly — there is no keda binary to learn.

Gotchas and failure modes

☺ Like you’re 10: Here are the ways KEDA looks fine and quietly does nothing — or does the opposite of what you meant.

Two autoscalers fighting over one workload

This is the classic. KEDA creates and owns the generated HPA, so writing your own HPA against the same Deployment leaves two controllers writing spec.replicas from different opinions — the workload oscillates, and neither reports an error, because each is doing its job. The admission webhook catches this in one direction only: it validates KEDA’s own resources, so applying a ScaledObject at a workload that already has an HPA is rejected at kubectl apply — but nothing stops someone hand-creating a rogue HPA afterwards, and even the one-directional check needs the webhook installed and reachable. The rules: one autoscaler per target, and never hand-edit keda-hpa-* — the operator reconciles your edit away. To change HPA behaviour, set advanced.horizontalPodAutoscalerConfig.behavior on the ScaledObject. Migrating an existing workload means deleting the old HPA first.

The workload never wakes up, or never sleeps

Work the chain in order. Is ACTIVE False while the queue genuinely has messages? Then the scaler is reading a different queue, the credentials are wrong, or activationThreshold sits above the current depth — the operator logs name the scaler and the error. Is READY False? The trigger metadata failed to parse, or the endpoint is unreachable. Does the queue look empty to KEDA but full to you? Some scalers count only visible messages, so a backlog stuck in flight or in a dead-letter queue reads as zero.

The mirror-image bug is a workload that refuses to sleep. Almost always the source is never truly quiet — a heartbeat or a monitoring probe on the same queue — and with the default activationThreshold: 0, any non-zero value counts as activity. Raise activation above the noise floor. Second suspect: cooldownPeriod is longer than you remember, and the clock restarts on every blip.

Reaction time, and the two intervals

pollingInterval is the operator’s loop and only really governs the zero boundary — waking from zero, and the check that lets a quiet workload return to zero. Above zero the HPA controller drives, at its own sync period of roughly 15 seconds, which KEDA cannot change — it is a kube-controller-manager flag. Dropping pollingInterval to 5 to “scale faster” therefore does nothing for a running workload while multiplying load on the broker. Add the HPA’s default five-minute scale-down stabilisation window and the honest story is: waking from zero costs a polling interval plus a pod start, scaling up from N takes tens of seconds, and scaling down is deliberately slow. If a spike is faster than that, you need a warm floor, not a smaller interval.

Everything downstream of the replica number

KEDA changes a number; the cluster has to honour it. A ScaledObject asking for 200 replicas on a full cluster yields 200 Pending pods until Karpenter or the Cluster Autoscaler adds nodes, so the real latency of a burst includes node provisioning. A ResourceQuota or a tight PodDisruptionBudget caps or blocks scaling with the error buried in the ReplicaSet, not the ScaledObject. And 200 replicas means 200 new connections to your database or broker — the bottleneck simply moves. The troubleshooting playbook has the decision tree; the short version is that “KEDA isn’t scaling” is usually a Pending-pod problem wearing a costume.

Alternatives and when to choose it

☺ Like you’re 10: Other things also decide “how many copies,” but they listen to different sounds — CPU, requests, or the clock.

Autoscaling choices are best made by asking one question: what is the honest signal that you are behind? If it is CPU, use the HPA and stop. If it is a backlog you can count, KEDA. If it is inbound HTTP requests, the request-driven options. Everything else follows.

The comparison that decides it

A practical rule

Reach for KEDA when a workload consumes from something that buffers and can be counted. Keep the plain HPA for anything CPU-shaped. Use minReplicaCount: 0 freely in dev, staging and internal batch — that is where the savings live and where the cold start costs nobody — and keep a warm floor of one or two on hot user-facing paths. Prefer ScaledJob when a unit of work is a discrete execution and ScaledObject when it is a long-running consumer. Then confirm the win in observability data rather than assuming it, and see The Tool Landscape for how KEDA sits among the other projects on this site.

Exam relevance and going further

☺ Like you’re 10: KEDA isn’t on the exam’s tool list, and you can’t open its website on exam day — so learn the idea, not the field names.

KEDA does not appear on the official CNPE tool list, so no task will ask you to author a ScaledObject from scratch. What the exam does test is the surrounding understanding: which autoscaler changes what, why an HPA cannot reach zero, what an external metric is, and how an operator extends the cluster through CRDs. KEDA is the clearest example of all four, which is why it earns a page. If a scenario mentions queue-driven or scale-to-zero workloads, the expected reasoning is “HPA plus an external metrics source, with something owning the zero boundary.”

The documentation allowlist — read this twice

⚖ CNPA vs CNPE — That allowlist mechanic is CNPE-specific — CNPA is fully closed-book, with zero external documentation, zero lookups of any kind, and no allowlist at all, which makes it stricter than CNPE, not looser. Even so, the concept-level knowledge here — what an HPA can’t do on its own and what an operator like KEDA adds — is still worth having memorized for CNPA’s closed-book recall.

What to be able to say without notes

Explain in one breath that KEDA generates and owns an HPA and only handles 0↔1 itself. State why an HPA cannot scale to zero (its algorithm scales the current replica count, and minReplicas is at least 1 by default). Distinguish threshold — the target value per replica that drives the HPA formula — from activationThreshold, the on/off switch out of zero. Distinguish pollingInterval (the operator’s loop, which matters at zero) from the HPA’s ~15-second sync period (which matters above zero). Name the three components and say which one fails in which symptom. Say why ScaledJob exists and when you would prefer it. And be able to state the precondition for scale-to-zero: the source must buffer, and the consumer must be idempotent. The wider autoscaling picture — HPA, VPA, Cluster Autoscaler, Karpenter, requests and limits, QoS — is in Scaling & Scheduling, and the vocabulary is in the glossary.

Official resources for after the exam

Outside the exam, the canonical sources are keda.sh/docs (the Scalers catalogue is the page you will actually live in), the ScaledObject and ScaledJob specifications under Concepts, the project at github.com/kedacore/keda, the HTTP Add-on at github.com/kedacore/http-add-on, and cncf.io/projects/keda. For the Kubernetes half — which is on the exam allowlist — read the HorizontalPodAutoscaler walkthrough on kubernetes.io until the scaling formula and the behavior block feel obvious.