CircleCI

CircleCI is a dedicated continuous-integration and delivery SaaS: a pipeline definition file, .circleci/config.yml, that turns every push into jobs, packaged reuse called orbs, and a specific, well-earned reputation for making Docker-heavy pipelines fast through layer caching and horizontal test splitting. It predates GitHub Actions by several years and was built on a premise that still shapes how it's sold today: CI/CD is its own product, not a feature bolted onto wherever your code happens to live. This page covers the config anatomy you'll actually write, the orb ecosystem that makes CircleCI feel less like YAML and more like a package manager, the performance mechanics that are the tool's real selling point, and an honest comparison against the SCM-bundled competitors that have eaten a lot of its market since 2019.

What CircleCI is and the problem it solves

☺ Like you're 10: It's a CI service that lives outside GitHub or GitLab on purpose, so switching where your code is hosted doesn't force you to also rewrite how it gets built and tested.

CircleCI launched in 2011, years before GitHub Actions (2019) or GitLab CI/CD's YAML-native pipelines matured, and its founding bet was that build-and-test automation deserved to be a standalone product with its own UI, its own insights, and its own pricing — connected to whichever source control host you use rather than shipped as a feature of one. It still works that way: you connect a GitHub, GitLab, or Bitbucket organization (confirm current VCS support on CircleCI's own docs, since integrations have shifted over the product's history), CircleCI listens for webhooks, and every push or pull request that touches a branch with a .circleci/config.yml file triggers a pipeline — a single execution of that file, with its own numbered ID, that fans out into one or more workflows, each of which runs one or more jobs.

That's the whole mental model: config declares jobs (what to run), workflows (in what order, on what trigger, with what gating), and — as of config version 2.1 — orbs (where to borrow both from). Nothing here is exotic compared to GitHub Actions or GitLab CI/CD; the pitch is less about a novel execution model and more about doing the boring parts — caching, parallel test execution, dependency-free portability across SCM providers — unusually well, plus a genuinely large public library of pre-built orbs going back over a decade.

Anatomy of config.yml: jobs, steps, executors, workflows

☺ Like you're 10: One file says what to run (jobs), what environment to run it in (executors), and in what order with what conditions (workflows) — everything else is decoration on those three ideas.

Everything lives in .circleci/config.yml at the repository root. version: 2.1 is the line that unlocks orbs, reusable commands, reusable executors, and matrix jobs — there is no reason to write anything older today. A minimal but realistic config for a Node service looks like this:

version: 2.1

orbs:
  node: circleci/node@5.1.0          # namespace/orb@semver — pulled from the Orb Registry
  aws-cli: circleci/aws-cli@4.1.3

executors:
  node-small:
    docker:
      - image: cimg/node:20.11       # CircleCI's own "convenience images" — small, fast to pull
    resource_class: small

jobs:
  lint:
    executor: node-small
    steps:
      - checkout                      # a built-in step: clone the repo at this commit
      - node/install-packages:        # a STEP contributed by the node orb, namespaced orb/command
          pkg-manager: npm
      - run: npm run lint

  test:
    executor: node-small
    parallelism: 4                    # split this job across 4 containers
    steps:
      - checkout
      - node/install-packages
      - run:
          name: Run split test suite
          command: |
            TESTFILES=$(circleci tests glob "test/**/*.spec.js" | circleci tests split --split-by=timings)
            npx jest $TESTFILES
      - store_test_results:
          path: test-results           # feeds CircleCI's Insights / flaky-test dashboard

  build-and-push:
    docker:
      - image: cimg/base:2024.01
    resource_class: medium
    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: true   # see the performance section below
      - run: docker build -t acme/checkout:${CIRCLE_SHA1} .
      - aws-cli/setup
      - run: |
          aws ecr get-login-password | docker login --username AWS --password-stdin $ECR_REGISTRY
          docker push acme/checkout:${CIRCLE_SHA1}

workflows:
  build-test-deploy:
    jobs:
      - lint
      - test:
          requires: [lint]            # fan-out/fan-in: test only starts once lint passes
      - build-and-push:
          requires: [test]
          filters:
            branches: { only: main }  # only build+push images from main
          context: aws-prod-creds     # injects org-level secrets scoped to this context

A few pieces are worth naming explicitly because they trip people up later. A job is a list of steps that runs in one execution environment, chosen by an executor — docker (a container, the default and cheapest), machine (a full VM with Docker pre-installed, no setup_remote_docker needed), macos (for iOS builds), or windows (via the circleci/windows orb). A workflow orchestrates jobs with requires (dependency graph, not file order — CircleCI parallelizes anything not blocked by a requires) and filters (branch/tag gating). resource_class picks how much CPU/RAM the job gets — small through 2xlarge for Docker, plus GPU and ARM classes — and it is the single biggest lever on both job speed and the credit bill, so treat it as a real tuning knob, not a default to ignore.

Orbs: reusable workflow packages

☺ Like you're 10: Orbs are recipe cards other people already wrote and tested — instead of retyping "install Node, restore the cache, run npm ci" in every repo, you write one line that pulls in someone else's version of that recipe.

An orb is a versioned, publishable package of commands, jobs, and executors that you reference with namespace/orb-name@version, addressed at the CircleCI-run Orb Registry (circleci.com/developer/orbs). Declaring orbs: { node: circleci/node@5.1.0 } makes every command, job, and executor inside that orb available under the node/ prefix — node/install-packages, node/test, and so on — the same way an npm package exposes functions under its module name. This is the single biggest reason CircleCI configs tend to be shorter than the equivalent hand-rolled Jenkins pipeline: nobody at Acme has to remember the right dependency-cache key format for npm, because the node orb's maintainers already worked that out and shipped it as a tested, versioned command.

Orbs come in three trust tiers, and the distinction matters for security posture, not just convenience. Certified orbs are authored and maintained by CircleCI itself (the circleci/ namespace — node, aws-cli, docker, slack). Partner orbs are authored by the vendor they integrate with and reviewed by CircleCI (Datadog's, Snyk's, HashiCorp's own orbs). Community orbs are published by anyone with a CircleCI account and carry no review at all. Organizations can — and security-conscious ones should — restrict uncertified orb usage in Organization Settings, because an uncertified orb is arbitrary code with access to your build environment and, if a workflow attaches a context, potentially your secrets.

Writing and publishing your own orb

Platform teams write internal orbs for the same reason platform-engineering teams write Helm library charts: to turn a house convention — the standard security scan step, the standard release-tagging logic — into something a version bump distributes rather than a wiki page nobody reads. The workflow is circleci orb init to scaffold commands/, jobs/, and examples/ as separate YAML files, circleci orb pack src > orb.yml to flatten them into one publishable file, circleci orb validate orb.yml, and circleci orb publish orb.yml myorg/deploy@dev:first to push a dev version for testing before circleci orb publish promote myorg/deploy@dev:first patch cuts the first real release.

The performance angle: Docker Layer Caching, resource classes, and test splitting

☺ Like you're 10: Three separate tricks — remembering unchanged parts of a Docker image, giving a job a bigger machine, and splitting one huge test suite across several machines at once — are what CircleCI is best known for.

CircleCI's marketing has leaned on speed since the beginning, and three specific mechanisms back that up. They solve different problems and are frequently confused with each other, so it's worth keeping them separate.

Docker Layer Caching (DLC)

DLC only matters for jobs that build a Docker image as part of the pipeline — publishing your own application image, not the executor image your job runs inside. Because a job running in the Docker executor is itself a container, it cannot safely run privileged Docker-in-Docker to build further images; setup_remote_docker instead hands the job a separate, isolated remote Docker Engine to talk to over the network. Add docker_layer_caching: true to that step and CircleCI reuses image layers from a previous build's remote Docker environment, so a docker build that only changed application code re-executes just the final COPY/RUN layers instead of reinstalling the entire OS and dependency stack. The machine executor gets the same benefit without needing setup_remote_docker at all, since it hands you a real VM with Docker already running natively. This is historically a Performance-tier-and-above feature, not available on every plan — confirm current plan gating on CircleCI's pricing page before you design a pipeline around it.

Resource classes and parallelism

resource_class is the straightforward lever: more vCPU and memory per job, at a proportionally higher credit cost per minute, available in classes from small up through 2xlarge for Docker (plus dedicated ARM and GPU classes for cross-architecture builds and ML workloads). parallelism: N is the more interesting one — it spins up N identical containers for a single job and expects you to divide the work between them, most commonly test files via circleci tests split. The --split-by=timings flag is the detail that makes this genuinely good rather than just "run it N times faster if you're lucky": CircleCI records how long each test file took on prior runs (fed by store_test_results) and splits the suite into N groups of roughly equal wall-clock time, not equal file count — which matters, because a naive alphabetical or count-based split routinely leaves one container running for ten minutes after the other three finished in two.

Day-to-day: the CLI, contexts, and dynamic config

☺ Like you're 10: A handful of commands cover almost everything — check the recipe makes sense, try it on your own laptop first, and keep the secrets in a shared drawer instead of writing them into the recipe itself.

# validate config without touching the API — catches YAML and orb-reference errors fast
$ circleci config validate

# fully resolve every orb reference into the literal expanded YAML — read this when
# an orb's behavior is confusing; it turns namespace/command into what it actually runs
$ circleci config process .circleci/config.yml

# run a job on your own machine before pushing — DOCKER EXECUTOR JOBS ONLY,
# no machine/macos/windows support, and no workflow orchestration (one job at a time)
$ circleci local execute --job test

# scaffold, pack, and publish an orb
$ circleci orb init myorg/deploy
$ circleci orb pack src > orb.yml
$ circleci orb publish orb.yml myorg/deploy@dev:first
$ circleci orb publish promote myorg/deploy@dev:first patch

# split tests by prior timing data — the command actually used inside a job, not locally
$ circleci tests glob "test/**/*.spec.js" | circleci tests split --split-by=timings

Contexts are how secrets and shared env vars reach a job without living in config.yml: created once in Organization Settings, referenced by name from any workflow with context: aws-prod-creds, and — critically — restrictable to specific security groups so a context holding production deploy credentials isn't automatically available to every repository in the org. That's the same job HashiCorp Vault does for runtime secrets, applied to build-time secrets instead — see Secrets & Credential Management for the broader pattern. For monorepos, dynamic config (a top-level setup: true plus the circleci/continuation orb) lets a small initial pipeline inspect what actually changed and generate the real config on the fly — so a change to one microservice's directory doesn't trigger every other service's test suite.

Gotchas and failure modes

☺ Like you're 10: Most of the pain comes from three things: forgetting the environment resets between steps, floating on an orb version that changes under you, and secrets ending up somewhere they shouldn't.

The other recurring trap is caching's evil twin: a dependency cache keyed too loosely (say, on branch name instead of lockfile hash) silently serves stale dependencies for months until a version-specific bug shows up in production that nobody can reproduce locally, because the CI environment and the developer's laptop have quietly diverged. Key caches on the lockfile's checksum, always — {{ checksum "package-lock.json" }} — never on something that doesn't change when the dependency tree does.

CircleCI vs GitHub Actions vs GitLab CI/CD vs Jenkins

☺ Like you're 10: If your code already lives in one place and you're happy staying there, the built-in tool is simpler; if you want the CI layer to survive a change of address, a decoupled tool like CircleCI is the point.

The decision that actually matters is rarely "which YAML syntax is nicer" — it's whether you want your CI/CD layer coupled to your source host or independent of it.

In practice the strongest case for CircleCI specifically shows up in three shapes: a company mid-acquisition running both GitHub and GitLab repos that wants one CI tool and one dashboard across both; a platform team that has accumulated years of institutional orb tooling and doesn't want to rewrite it against GitHub Actions' composite-action model; and any team whose pipelines are dominated by Docker image builds, where DLC and mature parallel test splitting are a measurable wall-clock win over a from-scratch GitHub Actions cache setup. Teams that are simply "on GitHub and happy there" increasingly default to GitHub Actions precisely because the integration cost of a second vendor stops paying for itself — which is the honest reason CircleCI's public messaging leans so hard on decoupling and performance rather than novelty of features.