OpenCost

OpenCost is the CNCF project that answers the one question a cloud bill physically cannot: who on this shared cluster actually spent the money? It watches every pod through Prometheus, works out how much CPU, memory, GPU, disk and network each one held, multiplies that by the real price of the node it was sitting on, and hands you a per-namespace, per-controller, per-label, per-pod breakdown through an API, a UI, a kubectl plugin, and Prometheus metrics. It solves the platform problem of cost being a single monthly number that nobody owns, nobody can act on, and everybody argues about — by turning it into a measured, attributable, dashboardable signal like any other.

What OpenCost is and the problem it solves

☺ Like you’re 10: Your cloud bill says “one big computer, $500.” OpenCost tells you which teams were sitting on that computer, and how much of it nobody used at all.

OpenCost began life as the open-source measurement core of Kubecost, was contributed to the CNCF by Stackwatch in 2022, and is now the community-maintained specification and reference implementation for Kubernetes cost monitoring. That word “specification” matters: it is a published, vendor-neutral definition of how a shared cluster’s cost should be attributed — which is why the same vocabulary (allocation, idle, shared, efficiency) turns up in Kubecost, in cloud vendors’ Kubernetes cost views, and in half the FinOps dashboards you will ever meet.

The problem: the invoice knows nodes, not tenants

Cloud providers bill by infrastructure: so many hours of m6i.4xlarge, so many GB-months of disk, so many GB of egress. That is a fine invoice when one application owns one machine. On a shared Kubernetes cluster it collapses, because a single node simultaneously runs the payments team’s API, the search team’s indexer, three platform DaemonSets, and a large hole of capacity nobody claimed. The invoice knows the node cost the way a restaurant bill knows the table total. Disaggregating “node-hours” into “team X spent $Y this week” is the entire technical problem, and OpenCost exists to solve it.

The allocation model in one sentence

Here is the whole idea, and it is worth memorising: for every container, over every time window, OpenCost computes cost = max(requested, used) × the node’s hourly rate for that resource × hours, separately for CPU, memory and GPU, then adds storage priced on the requested capacity of each PersistentVolume plus load-balancer and network costs. The max() is the part people trip over and the part that changes behaviour. A resource request reserves capacity on a node that the scheduler will not give to anyone else, so you are paying for it whether your process touches it or not. Ask for 4 CPUs and use 0.4, and OpenCost charges you for 4 — correctly, because that is what the cluster actually spent on your behalf.

What OpenCost deliberately is not

OpenCost is not an accounting system, not an invoice, and not a long-term data warehouse. It computes allocations over the window your Prometheus can still answer questions about, and it does not reconcile to the cent against your provider’s monthly statement — refunds, credits, support charges, marketplace fees and amortised commitments all live outside its model. It also does not act: it will not resize a Deployment, delete an idle namespace, or buy you a Savings Plan. Treat its output as a decision-grade estimate, not a ledger.

Where OpenCost fits in a platform

☺ Like you’re 10: It lives in the watchtower next to the other measuring tools — it doesn’t build or deploy anything, it just counts, in money.

In the reference architecture, OpenCost sits in the observability plane as a platform-team-operated shared service, with a foot in the governance plane. It is deployed once per cluster by the platform team through GitOps like any other add-on, and every tenant consumes it without installing anything. Nothing about it is tenant-specific except the labels your workloads already carry.

Its neighbours, and who does what

Prometheus is the substrate OpenCost reads from and writes back to — it is both an input (container CPU and memory series) and an output (OpenCost exports cost as metrics). Grafana renders those metrics; OpenCost stores no dashboards of its own worth defending. Karpenter and the Cluster Autoscaler are the tools that shrink the idle line OpenCost exposes, and bin-packing and autoscaling is where its findings turn into savings. Kubecost is the commercial product built on top of it, adding retention, multi-cluster aggregation, savings recommendations and SSO. And Kyverno or Gatekeeper are how you stop the waste coming back — by requiring resource requests and an owner label on every workload, so nothing lands in the dreaded __unallocated__ bucket.

CNPE domain relevance

OpenCost is on the official CNPE tool list, and cost sits inside Observability & Operations — the exam’s 20% domain — because on a real platform cost is just another operational signal with dashboards, alerts and owners. It also brushes Platform Architecture (capacity and node-shape decisions), self-service (a tenant seeing their own spend in the portal), and governance (budgets and quotas as guardrails). Study it alongside the FinOps deep dive, which is the lesson this page belongs to.

How it works — architecture, components, resources

☺ Like you’re 10: One small program joins three lists together: what every pod held, what every node costs, and what the cloud actually charged you.

OpenCost is refreshingly small: a single Go binary running as one Deployment, usually with an optional second container serving a static web UI. It has no custom resources of its own — no CRDs to learn, nothing for the API server to validate. Everything is configured through Helm values, environment variables, a pricing ConfigMap, and a Secret holding cloud credentials. That is unusual on this tool list and worth remembering: if an exam-style task asks you to configure OpenCost, you are editing a ConfigMap or a Deployment, not authoring a custom resource.

The three inputs

The binary continuously joins three streams. First, the Kubernetes API, which it watches directly with client-go for pods, nodes, PVs, PVCs, Services and controllers — this is where labels, ownership and topology come from, and it is why OpenCost does not strictly require kube-state-metrics. Second, Prometheus, for the time-series it cannot get from the API: container CPU usage (container_cpu_usage_seconds_total) and memory working set (container_memory_working_set_bytes), both originally scraped from the kubelet’s cAdvisor endpoint. Requests come from the pod specs OpenCost already watches on the API server, which is why usage — not requests — is the part that breaks when Prometheus is misconfigured. Third, a pricing source: a cloud billing integration, the provider’s public pricing API, or a custom pricing file you write yourself.

Node pricing, spot, and the on-prem case

To price a node, OpenCost reads its labels — node.kubernetes.io/instance-type, topology.kubernetes.io/region, and the provider’s capacity-type label (for example eks.amazonaws.com/capacityType, cloud.google.com/gke-preemptible, or karpenter.sh/capacity-type) — and looks up a rate. With a full billing integration (an AWS Cost and Usage Report in S3, a GCP BigQuery billing export, or an Azure billing export) it uses your negotiated, discounted, spot-adjusted rates. Without one it falls back to public list prices, which are wrong for anyone with a discount. On bare metal there are no rates to look up at all, so you supply them: amortise your hardware into an hourly cost per CPU-core, per GB of RAM and per GB of storage, and hand OpenCost that file. It then splits each node’s total cost across CPU and RAM in proportion to those rates and divides the result among containers by the allocation formula.

The three buckets: allocated, idle, shared

Every dollar of node cost lands in exactly one of three buckets, and how you treat the last two is a genuine design decision with political consequences.

The resources you will actually write

☺ Like you’re 10: Three little files: where to find Prometheus, what your hardware costs, and what to put on the dashboard.

1. Installing it — the Helm values that matter

OpenCost is installed from its Helm chart, and roughly four values decide whether it works. It must know how to reach Prometheus, what to call this cluster, and (optionally) where the cloud billing credentials live. Note there is no CRD in sight — this is a Deployment, a ServiceAccount with read-only RBAC, a Service on ports 9003 (API and metrics) and 9090 (UI), and some ConfigMaps.

# values.yaml for the opencost/opencost chart — applied via GitOps like any add-on.
opencost:
  exporter:
    defaultClusterId: prod-eu-west-1      # shows up as the cluster label everywhere
    resources:
      requests: { cpu: 10m, memory: 55Mi }   # practise what you preach 🦥
      limits:   { memory: 1Gi }
    # Cost metrics are exported on /metrics by default — that exposition is the
    # integration point for Grafana, alerting rules, and anything speaking PromQL.
    # extraEnv is where you set OpenCost's own environment variables, e.g. to also
    # emit pod annotations as a metric so you can aggregate by annotation.
    extraEnv:
      EMIT_POD_ANNOTATIONS_METRIC: "true"

  prometheus:
    # EITHER an in-cluster Prometheus...
    internal:
      enabled: true
      serviceName: kube-prometheus-stack-prometheus
      namespaceName: monitoring
      port: 9090
    # ...OR an external / remote one:
    # external:
    #   enabled: true
    #   url: "https://mimir.acme.dev/prometheus"

  ui:
    enabled: true            # the small built-in web UI on :9090

  cloudIntegrationSecret: cloud-integration   # Secret holding the CUR / BigQuery config

  # Let the Prometheus Operator discover OpenCost's own /metrics endpoint.
  # (The ServiceMonitor lives under opencost.metrics in this chart, not at the root.)
  metrics:
    serviceMonitor:
      enabled: true
      additionalLabels:
        release: kube-prometheus-stack   # the adoption label the Prometheus CR selects on

2. Custom pricing — the on-prem and “our discounts are secret” case

When there is no cloud billing API — bare metal, a private cloud, or a heavily negotiated contract you would rather not expose — you tell OpenCost the rates yourself. The prices are hourly, per unit: per CPU core-hour, per GB of RAM per hour, per GB of storage per hour. Derive them by amortising hardware over its depreciation life plus power, cooling, rack and staff, then dividing by capacity. Everything downstream — allocation, idle, efficiency, showback — flows from these numbers, so document how you computed them.

# The rate card is a JSON document keyed default.json inside a ConfigMap that the
# chart mounts for you (opencost.customPricing.* switches OpenCost over to it).
# Every value is a STRING and every rate is hourly, per unit:
#   CPU / spotCPU        per vCPU-hour   (no spot on bare metal, so they match)
#   RAM / spotRAM        per GB-hour
#   GPU                  per GPU-hour
#   storage              per GB-hour of PROVISIONED PV capacity
#   *NetworkEgress       per GB transferred
# Note JSON has no comments — keep the block below free of them or it won't parse.
apiVersion: v1
kind: ConfigMap
metadata:
  name: custom-pricing-model
  namespace: opencost
data:
  default.json: |
    {
      "description": "acme on-prem DC1 - 5yr amortised hardware + power + rack",
      "CPU": "0.021000",
      "spotCPU": "0.021000",
      "RAM": "0.002800",
      "spotRAM": "0.002800",
      "GPU": "0.640000",
      "storage": "0.000045",
      "zoneNetworkEgress": "0.000000",
      "regionNetworkEgress": "0.000000",
      "internetNetworkEgress": "0.045000"
    }

3. The metrics OpenCost emits, and what to build on them

OpenCost’s Prometheus exposition is the most important integration surface, because it lets cost join every tool you already run. The metric names below are the ones you will actually query; note that the *_allocation series are resource allocations (cores, bytes) which you multiply by the node’s hourly cost series to get money.

# --- what OpenCost exports (the ones worth knowing) ---
#   node_cpu_hourly_cost{node,instance_type,provider_id}   $ per core-hour
#   node_ram_hourly_cost{...}                              $ per GB-hour
#   node_gpu_hourly_cost{...}   node_total_hourly_cost{...}
#   container_cpu_allocation{namespace,pod,container,node} cores held = max(req,usage)
#   container_memory_allocation_bytes{...}
#   container_gpu_allocation{...}   pv_hourly_cost{...}   pod_pvc_allocation{...}
#   kubecost_load_balancer_cost{...}   kubecost_cluster_management_cost{...}

# Hourly CPU spend per namespace: allocation x the rate of the node it sits on.
sum by (namespace) (
    sum by (namespace, node) (container_cpu_allocation)
  * on (node) group_left() avg by (node) (node_cpu_hourly_cost)
)

# Cluster CPU efficiency: how much of what you PAID FOR you actually USED.
  sum(rate(container_cpu_usage_seconds_total{container!=""}[1h]))
/ sum(container_cpu_allocation)

# Idle CPU cost per hour: node capacity nobody claimed, priced.
# (kube_node_status_capacity comes from kube-state-metrics, not from OpenCost.)
  sum(kube_node_status_capacity{resource="cpu"} * on (node) group_left()
      avg by (node) (node_cpu_hourly_cost))
- sum(container_cpu_allocation * on (node) group_left()
      avg by (node) (node_cpu_hourly_cost))

# Projected monthly CPU spend for one namespace: hourly cost x 730 h/month.
# Allocation alone is CORES, not money — you must multiply by the node rate first.
sum(
    sum by (namespace, node) (container_cpu_allocation{namespace=~"payments.*"})
  * on (node) group_left() avg by (node) (node_cpu_hourly_cost)
) * 730

Wrap the second one in a PrometheusRule and you have a cost alert that behaves like any other SLO breach — “namespace efficiency below 20% for 24 hours” is a far better trigger for a right-sizing conversation than a monthly finance email. See Prometheus for the rule syntax and Observability for where it belongs.

Day-to-day commands

☺ Like you’re 10: Mostly you ask it one question — “what did this cost, split up how?” — through a URL or a kubectl command.

The allocation API

Everything the UI and the plugin do is one HTTP call. Learn the four parameters and you can answer almost any cost question from a terminal: window (how far back — 7d, 24h, or explicit dates), aggregate (how to slice — namespace, controller, pod, label:team, or a comma-separated combination), accumulate (one total for the window, or one row per day), and the idle/shared switches.

kubectl -n opencost port-forward svc/opencost 9003 9090   # API on 9003, UI on 9090

# Cost by namespace over the last week, one total per namespace.
curl -sG http://localhost:9003/allocation/compute \
  -d window=7d -d aggregate=namespace -d accumulate=true | jq '.data'

# Slice by a team label instead — this is what showback actually runs on.
curl -sG http://localhost:9003/allocation/compute \
  -d window=30d -d aggregate=label:team -d accumulate=true | jq

# Daily series for one namespace, including the idle bucket as its own row.
curl -sG http://localhost:9003/allocation/compute \
  -d window=14d -d aggregate=namespace -d accumulate=false \
  -d includeIdle=true -d idleByNode=true | jq '.data[] | keys'

# Distribute idle back onto tenants instead of showing it separately.
curl -sG http://localhost:9003/allocation/compute \
  -d window=7d -d aggregate=namespace -d shareIdle=true | jq

# The other endpoint: assets (nodes, disks, load balancers) rather than workloads.
curl -sG http://localhost:9003/assets -d window=7d | jq '.data | keys'

curl -s http://localhost:9003/metrics | grep -E '^node_.*_hourly_cost' | head
curl -s http://localhost:9003/healthz

The kubectl cost plugin

kubectl-cost (install via krew) is the friendly front door. It defaults to talking to Kubecost — the kubecost namespace, the cost-analyzer Service, port 9090 — so against a plain OpenCost install you must point it at the right namespace, Service, port and API path. Forgetting one of those four flags is the single most common reason it appears “broken”.

kubectl krew install cost

# Against a standalone OpenCost, always pass these four (namespace, Service,
# port, API path) — the defaults all point at a Kubecost install instead.
OC="--kubecost-namespace opencost --service-name opencost"
OC="$OC --service-port 9003 --allocation-path /allocation/compute"

kubectl cost namespace  $OC --window 7d --show-efficiency
kubectl cost controller $OC --window 30d --show-cpu --show-memory --show-pv
kubectl cost pod        $OC --window 24h --namespace payments
kubectl cost label      $OC --window 30d -l team          # showback by team label
# Want machine-readable output for a report? Call /allocation/compute with curl
# (above) and pipe it through jq — the plugin's job is a readable table.

# Sanity-check the inputs when numbers look wrong.
kubectl -n opencost logs deploy/opencost -c opencost | grep -i 'pricing\|prometheus'
kubectl get nodes -L node.kubernetes.io/instance-type,topology.kubernetes.io/region
kubectl get nodes -L karpenter.sh/capacity-type      # is spot being detected?
kubectl -n opencost get cm                            # pricing + cloud integration config

Gotchas and failure modes

☺ Like you’re 10: The dangerous failure isn’t “no numbers.” It’s confident wrong numbers that people then make decisions with.

The Prometheus dependency is load-bearing

OpenCost is only as good as the metrics underneath it. Three things bite. If cAdvisor container metrics are being dropped by an over-eager metricRelabelings rule, usage goes to zero and every workload looks perfectly efficient. If your Prometheus retention is 15 days, a window=30d query cannot be answered and quietly returns a short or empty result rather than an error. And if the scrape interval is very coarse, short-lived pods and CronJobs are under-counted or missed entirely. Start every “these numbers look wrong” investigation in Prometheus, not in OpenCost — workload triage and the troubleshooting playbook use the same instinct.

List prices lie, and spot lies loudest

Without a cloud billing integration OpenCost prices nodes at public rates. If you have an enterprise discount, Savings Plans, Reserved Instances or Committed Use Discounts, every number will be systematically too high — and spot instances, whose real price moves hourly, will be badly wrong in whichever direction the market went. Worse, commitments are a financial construct with no cluster-visible footprint: a node covered by a three-year commitment looks identical to an on-demand one. Wire up the CUR / BigQuery export / Azure export before anyone is allowed to quote a figure in a meeting.

Unlabelled workloads become __unallocated__

Aggregating by label:team only works if the label exists. Every pod without it lands in an unallocated bucket that grows until it is the biggest line on the report and the whole exercise loses credibility. The fix is upstream, not in OpenCost: enforce an owner label at admission with Kyverno or Gatekeeper, put it in your chart or Kustomize base so it propagates to pods (labels on a Deployment do not automatically appear on its pods unless they are in the pod template), and make it part of the golden path in developer experience.

Scope, scale, and the sharp edges

Core OpenCost is single-cluster — no built-in federation, so a multi-cluster view means either Kubecost or exporting every cluster’s metrics to a central Prometheus-compatible store and aggregating yourself, which is also a multi-cluster conversation. It holds computed allocations in memory, so a restart re-derives from Prometheus and anything outside retention is gone for good. Its network egress figures are estimates unless you deploy additional network-cost collection. And exporting per-container cost series adds real cardinality to Prometheus: on a large cluster, aggregate before you store, or you will trade a cost problem for a monitoring-cost problem — a uniquely embarrassing outcome.

Alternatives and when to choose it

☺ Like you’re 10: Other cost tools exist. Most of the Kubernetes-aware ones either are OpenCost inside, or copy how it works.

OpenCost’s position is unusual: because it is the open specification as well as an implementation, several of its “competitors” consume or reimplement it. The choice is less “which engine” and more “how much product do you want wrapped around the engine.”

The practical rule: start with OpenCost in every cluster, because it is free, small, GitOps-friendly and vendor-neutral, and because getting labelling and idle policy right is the hard part regardless of the tool. Graduate to Kubecost or a SaaS only when a specific requirement arrives — usually multi-cluster rollups, thirteen months of retention, or per-customer unit economics the cluster alone cannot express. See The Tool Landscape for how OpenCost sits among the other named CNPE projects.

Exam relevance and going further

☺ Like you’re 10: On exam day you can’t open OpenCost’s website — so what it measures, and what its knobs mean, has to already be in your head.

OpenCost is named on the official CNPE tool list, and cost lives inside the Observability & Operations domain. It is more likely to appear as configuration, interpretation and reasoning than as a from-scratch install: point it at a Prometheus, read an allocation output and say which team is wasteful, explain why a namespace’s cost exceeds its usage, or fix a values file so a cluster gets the right ID and the right pricing source. Because it has no CRDs, there is nothing exotic to memorise — but you must know the model cold.

The documentation allowlist — read this twice

⚖ CNPA vs CNPE — That allowlist is a CNPE mechanic — CNPE is hands-on and at least lets you reach for kubernetes.io mid-task. CNPA is stricter still: it is fully closed-book multiple-choice, with zero lookups of any kind, so nothing here can be looked up there either. That said, knowing the allocation model cold is still worthwhile prep for CNPA’s closed-book recall.

What to be able to do without notes

State the allocation formula — max(request, usage) × node rate × hours — and explain why the max() is there. Define allocated, idle and shared cost, and describe two ways to handle idle and what each does to a team’s number. Explain the requests-versus-usage gap and name the two levers that close it (right-size the request; consolidate the nodes). Distinguish showback from chargeback and say which you start with. Name OpenCost’s three inputs and its four outputs (API, UI, kubectl cost, Prometheus metrics). Say what changes when there is no cloud billing integration, and what you do on bare metal. Know that OpenCost is the open core and Kubecost the commercial product on top. The wider CLI spine is in the command reference, and the surrounding practice tasks are on practice: observability.

Official resources for after the exam

Outside the exam, the canonical sources are opencost.io/docs (the Configuration and API sections are short and worth reading end to end), the source and Helm chart at github.com/opencost/opencost and opencost-helm-chart, the plugin at github.com/kubecost/kubectl-cost, the CNCF project page at cncf.io/projects/opencost, and the FinOps Foundation for the operating model around it. Pair this page with the FinOps deep dive for the full cost lifecycle, Prometheus for the metrics underneath, Karpenter and scaling & scheduling for shrinking the idle line, best practices for the habits, and the glossary whenever a term stops making sense.