Semgrep

Semgrep is an open-source static analysis engine whose whole pitch is that a rule should look almost exactly like the code it's looking for. Instead of a regex that breaks the moment someone reformats a file, or a heavyweight commercial engine that needs its own query language and a compiled model of your entire codebase before it can answer anything, Semgrep parses your source into a real abstract syntax tree and matches a rule — itself parsed the same way — directly against that structure. That single design choice is why it's become the default first SAST gate on most modern pipelines: fast enough to run on every pull request, and readable enough that an application team, not just a security specialist, can write and own its own rules.

What Semgrep is, and the problem it solves

☺ Like you're 10: It's the middle option between "dumb text search" and "an engine so deep it needs a PhD and an hour to run" — fast like the first, structure-aware like the second.

Semgrep (short for semantic grep) is an open-source, Apache-2.0-licensed static analysis CLI originally built by r2c, now developed by Semgrep Inc. It ships as a single pip-, brew-, or Docker-installable binary that needs no server, no database, and no network access to run: point it at a directory, give it a ruleset, and it parses every matching file, checks each rule's pattern against that file's structure, and prints findings. The matching engine itself — semgrep-core — is written in OCaml for speed; the CLI wrapper around it has historically been Python, with the project steadily moving more of that wrapper into the compiled core over time, so treat exact implementation-language claims as something to verify against the project's own repository rather than take as fixed.

The gap Semgrep fills is specifically the one between two bad options. A plain grep or regex-based linter has no idea what a comment, a string literal, or an argument boundary is, so it either misses reformatted code entirely or drowns real findings in false positives from a hit inside a code comment. At the other extreme, engines like CodeQL compile your entire codebase into a queryable relational database to support genuine interprocedural taint tracking — powerful, but slow enough that it doesn't belong on every commit, and its query language has a real learning curve. Semgrep sits deliberately in between: real parsers, real syntax trees, a rule syntax that reads like the target language itself, and a matching pass that's fast enough to gate a pull request without anyone noticing the wait. See SAST, DAST & SCA if you haven't already — it places static analysis as one of three pipeline scan categories; this page goes one tool deep.

Rule syntax: matching an AST, not compiling a whole-program database

☺ Like you're 10: A rule is a fill-in-the-blank version of the code you're worried about — Semgrep parses your real code the same way it parses the blank, then looks for a shape match.

A Semgrep rule is a YAML file with a pattern field, and that field is written in the target language itself, not in a separate query DSL. Write $CURSOR.execute($QUERY) as a Python pattern and Semgrep parses that string with Python's own grammar, producing a small pattern AST. It then walks the AST of every Python file in scope looking for a subtree with the same shape — same node types, same structure — ignoring whitespace, comments, and variable naming. A dollar-prefixed identifier like $QUERY is a metavariable: a wildcard that matches any single expression and binds to whatever it matched, so the same metavariable name reused later in the pattern ($X == $X) requires the same code to appear both times, not just any two expressions — that's the mechanism a rule uses to catch a self-comparison tautology, for instance. An ellipsis (...) matches zero or more of anything — arguments, statements, list elements — the difference between "this exact call" and "this call, with whatever else around it I don't care about."

Rules can go one step deeper than pure structural matching with taint mode (mode: taint), which adds a lightweight source-to-sink dataflow pass within a single file or function: you declare pattern-sources (where untrusted data enters), pattern-sinks (where it becomes dangerous), and optionally pattern-sanitizers (what neutralizes it in between), and Semgrep traces whether a source's value can reach a sink through local variable assignment without passing through a sanitizer — the same source/sink/sanitizer model covered generally in static analysis & secrets detection. That's real, and useful, but it's still a per-file (increasingly per-project with Semgrep's paid cross-file/cross-function analysis) pass, not the kind of true whole-program interprocedural taint tracking CodeQL gets by compiling your entire codebase into one relational database first. Keep that distinction precise: Semgrep's dataflow is an addition bolted onto AST pattern matching, not a different foundation underneath it.

The practical upshot: each file is a closed world. Semgrep never has to know what checkout.py imports from a module three directories away in order to match a rule against it, which is exactly what keeps a scan measured in seconds rather than minutes. It's also exactly the ceiling this page keeps returning to — the comparison against CodeQL and SonarQube later on is really a comparison of what each tool gave up to get its speed, or gave up its speed to get.

The community ruleset versus a rule you write yourself

☺ Like you're 10: Off-the-shelf rules catch things every codebase might have wrong; a rule you write yourself catches the one mistake that's specific to how your team's own code works.

Most Semgrep runs start with the free Registry at semgrep.dev/r — thousands of community- and vendor-contributed rules, organized into named packs you reference with a p/ prefix. semgrep scan --config=auto is the common default: it detects which languages and frameworks are present in the repository and pulls the relevant curated packs automatically, and if you're logged in via semgrep login, it also pulls whatever custom rules and policies your organization has configured centrally. Narrower, hand-picked packs are the more reproducible choice for a merge-blocking gate:

$ semgrep scan --config=p/ci .                 # curated, low-noise pack made specifically for CI gating
$ semgrep scan --config=p/owasp-top-ten .       # rules mapped to the current OWASP Top Ten categories
$ semgrep scan --config=p/security-audit .      # broader, higher-recall — expect more findings to triage
$ semgrep scan --config=p/python --config=p/flask .   # stack-specific packs, composable

The community rulesets are broad by necessity — written to be roughly right across every codebase that might install them — which means they can't see the one anti-pattern that only exists inside your own organization's own code. This course has a running example of exactly that gap: Secure Coding Patterns follows Benny building queries through an internal db.run() wrapper instead of the raw database driver, and Timmy's off-the-shelf ruleset never flags it, because it was trained to recognize the driver's own execute() call as a sink — it has no idea a wrapper three layers down is the same thing. That's not a bug in the ruleset; it's the exact shape of thing a generic pack structurally cannot know. The fix is a custom rule, and it's worth writing the fuller taint-mode version this time, so it also catches the unsafe value flowing through an extra layer of local variables before it ever reaches db.run():

# .semgrep/rules/acme-db-run-taint.yaml
# Off-the-shelf rules know the driver's execute() call as a sink.
# They've never been taught that OUR db.run() wrapper is the same thing.
rules:
  - id: acme-db-run-unsanitized-query
    mode: taint
    languages: [python]
    severity: ERROR
    message: >
      Untrusted input reaches db.run() without going through
      its own bind-parameter form. See go/db-wrapper.
    metadata:
      cwe: "CWE-89: SQL Injection"
      owasp: "A03:2021 - Injection"
      confidence: HIGH
    pattern-sources:
      - patterns:
          - pattern-either:
              - pattern: request.args.get(...)
              - pattern: request.form.get(...)
              - pattern: request.json[...]
    pattern-sinks:
      - patterns:
          - pattern: db.run($QUERY, ...)
          - pattern-not: db.run("...", ...)     # a pure string literal has nothing to inject
    pattern-sanitizers:
      - pattern: db.run($QUERY, params=$PARAMS)  # our wrapper's own bind-parameter form

Notice the shape of the fix: it's not "ban db.run()," which would just get the rule disabled by an annoyed team the first time it fires on a safe call — it's "teach the scanner that our own wrapper is a sink, and that our own wrapper's parameterized form is the sanitizer that makes it safe." That's the whole value proposition of writing your own rules instead of only running the Registry: the Registry knows every public framework's dangerous calls; only your team knows your team's. See static analysis & secrets detection for how a rule like this gets wired into a required, merge-blocking check, and for the baselining and confidence-threshold tuning that keeps a growing rule set from burying real findings under noise.

Day-to-day commands

☺ Like you're 10: One command to look, one command to gate a build, and a couple of ways to say "I already know about this one" without turning the rule off for everyone else.

# local, exploratory scan — good for authoring and testing a rule
$ semgrep scan --config=p/ci .
$ semgrep scan --config=.semgrep/rules/acme-db-run-taint.yaml .   # test one rule file directly

# semgrep ci — the CI-specific subcommand, not just "scan run in CI"
# diff-aware: only NEW findings on this branch, versus the baseline, are blocking
# logged-in orgs get their configured rulesets automatically, no --config needed
$ semgrep login
$ semgrep ci

# scope a scan to only what changed since a commit, without the ci subcommand's extras
$ semgrep scan --config=p/ci --baseline-commit=origin/main .

# machine-readable output for a dashboard, DefectDojo, or GitHub code scanning
$ semgrep scan --config=p/ci --sarif -o results.sarif .
$ semgrep scan --config=p/ci --json -o results.json .

# apply a rule's own fix: suggestion, not blind autofix — review the diff before it merges
$ semgrep scan --config=.semgrep/ --autofix --dry-run .

Two escape hatches matter for keeping a gate trustworthy rather than something people route around. A .semgrepignore file — same syntax as .gitignore, and Semgrep also respects .gitignore automatically — excludes whole paths: generated code, vendored dependencies, test fixtures deliberately full of "unsafe" examples. An inline // nosemgrep: acme-db-run-unsanitized-query (or # nosemgrep: rule-id, comment syntax per language) suppresses one specific line for one specific rule — always name the rule ID rather than a bare nosemgrep, which silences every rule on that line, including ones added after the comment was written and long forgotten.

Gotchas and failure modes

☺ Like you're 10: Most of the surprises come from the same root cause — a rule matches shape, not meaning, so it can be fooled by anything that looks right but isn't, or looks wrong but is actually fine.

The failure modes cluster around the same tradeoff the rest of this page keeps naming.

• Metavariable equality is syntactic, not semantic. $X == $X matches two textually identical expressions, not two expressions that are guaranteed to evaluate the same at runtime — get_user() == get_user() matches even if the function is non-deterministic. Useful for catching copy-paste tautologies; not a substitute for real value-flow analysis.
• Language depth is uneven. Semgrep's parser coverage spans 30-plus languages, but the maturity of that support — and of taint mode specifically — is meaningfully deeper for the languages the community and Semgrep Inc. invest the most in (Python, JavaScript/TypeScript, Java, Go) than for less common targets. Don't assume equal recall across every language in a polyglot monorepo; verify per-language coverage before trusting a "clean scan" as strong evidence.
• --config=auto isn't pinned. Registry rules change over time — a pack can gain or tighten a rule between two runs of the same pipeline with no code change at all, so a build that was green yesterday can start failing today on unrelated code. That's the same nondeterminism trap Helm's lookup function creates for GitOps reconcilers, just at the ruleset layer instead of the render layer. Pin a specific pack version, or vendor your chosen rules into the repo, if reproducibility matters more than always having the newest coverage.
• Taint mode still ends at the file/project boundary. It will not follow a tainted value across a network call into a different microservice, and by default it reasons within a single file or a project's own call graph, not the whole distributed system. A source-to-sink chain that crosses a service boundary needs a different kind of analysis entirely, not a stronger Semgrep rule.
• It is not a secrets scanner, even though it can pattern-match secret-shaped strings. A rule can flag something that looks like an AWS access key, but dedicated secrets tools like gitleaks and TruffleHog run entropy analysis and validity checks against git history itself, which Semgrep's file-scoped AST matching was never built to do. Keep the two disciplines separate rather than treating one clean Semgrep run as secrets coverage.

Where Semgrep sits: speed versus depth against CodeQL and SonarQube

☺ Like you're 10: Three tools, three different trades between how fast they run and how deep they actually think — and the right answer is usually more than one of them, at different points in the pipeline.

All three tools are legitimate SAST engines, and the real question is rarely "which one is best" — it's which stage of the pipeline each one earns its keep at.

A mature pipeline rarely picks just one. Semgrep runs first because it's cheap enough to run on literally every commit without anyone noticing the wait — that's the whole reason it exists. CodeQL earns its much higher cost on a schedule, or on the services where an interprocedural taint chain is actually worth the wait to catch, rather than on the hot path of every PR. SonarQube's Quality Gate workflow fits teams that want security findings sitting next to code-quality metrics in one dashboard, with Security Hotspots specifically designed to be triaged by a human rather than auto-blocking — a different philosophy from Semgrep's "clean means clean, findings mean fail" model. None of the three replaces the other two; each fills the gap the others' speed-versus-depth trade leaves open. See vulnerability management & triage for how findings from more than one of these get deduplicated into a single backlog once you're running more than one.