The Triage Playbook — diagnose a CDP challenge fast
Every CDP challenge drops you into a live environment with something already broken — a pipeline stage, a scanner config, a policy — and the clock is running from the moment you open it. The candidates who lose time aren't the ones who don't know the tools; they're the ones who trust a scan's output at face value and start "fixing" the wrong thing. A scan that reports zero findings looks identical to a clean codebase and to a scan that never actually pointed at your code. A DAST run that only covers the login page looks identical to an app with almost nothing to attack. A policy that blocks everything, including plans that are actually fine, looks identical to a strict security posture working as intended. This page exists to tell those apart fast: one diagnostic order to run before you form any opinion, and a symptom index that takes whatever's in front of you and tells you where the real problem almost always lives.
Picture two smoke detectors, both silent. One is silent because the room really is safe. The other is silent because somebody took the battery out. From across the room they look exactly the same — quiet. The only way to tell them apart is to walk over and check the battery. A scanner that reports zero findings looks exactly like a scanner that never actually pointed at your code. This whole page is the battery check.
The universal triage order
☺ Like you're 10: Before you guess what's wrong, always check the same five things first, in the same order. Guessing before you've looked is how six-hour windows disappear.
The single biggest time sink in a live-environment challenge is forming a hypothesis before reading the tool's own report of what it did. You see a clean scan, you decide the code must be fine, you move to the next challenge, and the scan never actually reached the file you were graded on. So: don't theorize. Run the sequence. It takes under two minutes and, more often than not, the tool has already told you the answer in its own output — you just have to read past the summary line.
The five steps, in order
Steps 1–3 apply to every scanner in the CDP toolchain, static or dynamic. Step 4 only matters for anything that talks to a live target — a running app, a cloud account, a cluster. Only at step 5 do you get to have an opinion about what's actually broken.
| # | Step | What you run | What you're looking for |
|---|---|---|---|
| 1 | Read the self-report | The tool's own summary line — every scanner prints one | "2 rules run on 0 files", "3 routes crawled", "0 packages found". This one line usually contains the whole diagnosis. |
| 2 | Confirm the target | The exact path, branch, image tag, or URL passed to the command | A scan of the wrong directory, a stale cached image, or a target still pointing at last challenge's staging URL. |
| 3 | Confirm the ruleset/policy loaded | --verbose / --debug and a rule count in the output | The config file that actually resolved, its rule count, and its version — not the one you assumed was wired in. |
| 4 | Confirm auth/session/credential state | A login indicator, a token, an sts:GetCallerIdentity equivalent | Anything that talks to a live app, cloud account, or cluster needs proof it's authenticated as something with real access — not a silent fallback to anonymous. |
| 5 | Hypothesis | — | Only now. Prefer the cause that explains every symptom, not just the loudest line in the output. |
# The two-minute opener. Run this shape on every scan, every time.
semgrep --config=auto --verbose . # 1. rule count + files scanned, printed up front
checkov -d . --compact --quiet 2>&1 | head -20 # 1. "Passed checks / Failed checks / Skipped" line
trivy image --debug registry.acme.io/app:1.4 2>&1 | tail -30 # 1+3. DB freshness, target resolved, rules applied
opa eval --data policy/ --input plan.json --explain=full \
'data.main.deny' | head -40 # 1+3. rule evaluation trace, not just pass/fail
trufflehog git file://. --since-commit main --only-verified # 2. confirm --since-commit isn't hiding old history
zap-baseline.py -t "$URL" -I 2>&1 | grep -i "routes\|spider" # 1+4. route count is the tell for a missing session
# Confirming the target itself, before trusting any of the above:
git rev-parse --abbrev-ref HEAD && git log -1 --oneline # right branch, right commit?
docker inspect --format '{{.Id}}' app:1.4 # is this the image you just built, or a cached one?
aws sts get-caller-identity # which identity is actually running this scan?Zero findings is not evidence of a clean codebase — it's evidence the tool ran. Those are different claims, and a scanner's exit code alone can't distinguish them: a config path that resolves to an empty directory, a ruleset that never loaded, and genuinely secure code all produce the identical output of "0 findings, exit 0." The self-report line — files scanned, rules loaded, routes crawled — is the only thing that tells them apart, which is exactly why step 1 comes before step 5, not after.
Symptom index
☺ Like you're 10: Find the words that match your screen in the left column. The middle column tells you what to test first. The right column says which page has the full story.
Once the five-step opener has produced evidence, this is the lookup — organized the way the CDP's own challenges are organized, from source code outward to the cloud account. Scan the left column for what you're actually staring at, test the middle column first, and follow the link for the full treatment.
| You see… | Most likely | Where to look |
|---|---|---|
| SAST scan reports zero findings | Wrong --config path resolved to nothing, or a .semgrepignore/exclude pattern silently covers the whole source tree | Static Analysis & Secrets Detection, Semgrep |
| SAST scan floods you with hundreds of findings | The community ruleset ran unscoped against generated code, vendored dependencies, or test fixtures full of deliberately "unsafe" examples | Static Analysis & Secrets Detection |
| SonarQube Quality Gate is green, but the bug is definitely there | Wrong project key analyzed on this run, or the "New Code" period is hiding a pre-existing issue outside its window | SonarQube |
| CodeQL database builds with a handful of source files | The build command in codeql database create doesn't match the project's real build system — most of the code was never compiled into the database | CodeQL |
| gitleaks/TruffleHog finds nothing on a repo you know has a leaked key | A diff-only or pre-commit scan never reaches history older than when scanning was enabled — needs a full, unscoped history run | Secrets Detection, gitleaks, TruffleHog |
TruffleHog reports the hit as Unverified | --only-verified is filtering it out, or that credential type has no live-verification detector — unverified still needs eyes on it | TruffleHog |
| SCA scan reports zero vulnerable dependencies | No lockfile present, so the scanner fell back to manifest-only resolution — or --scan's path skips node_modules/vendored deps entirely | Software Composition Analysis, Snyk, OWASP Dependency-Check |
| The same CVE keeps reappearing after you "fixed" it | You pinned the direct dependency; the vulnerable package is transitive, resolved three layers down the graph | Software Composition Analysis |
trivy image comes back clean on an image you know is vulnerable | Scanned a locally cached :latest tag instead of the image you just built, or the vulnerability DB was never downloaded (--skip-db-update left on by habit) | Trivy |
| Generated SBOM has zero or near-zero packages | syft pointed at the source directory instead of the built artifact — missing the docker: scheme prefix scans the wrong thing entirely | Syft & Grype |
cosign verify fails on an image you just signed | Verifying against the wrong public key or Fulcio identity, or the image was rebuilt after signing — a rebuild changes the digest even with identical source | Sigstore & cosign |
| DAST scan only covers the login page | No authentication context wired in — the spider never gets past the login shell, so the report looks clean because there's almost nothing left to attack | Dynamic Analysis in Practice, OWASP ZAP |
| DAST alert count quietly drops partway through a long scan | The authenticated session died mid-run (CSRF mismatch, idle timeout) with no loggedInRegex/loggedOutRegex verification configured to catch the fallback | Dynamic Analysis in Practice |
| DAST route count is tiny against a React/Vue/Angular target | Traditional link-only spider is blind to client-side routing — needs the AJAX spider (headless browser) or an OpenAPI/GraphQL import to seed real coverage | Dynamic Analysis in Practice, Burp Suite |
| A ZAP baseline scan finds almost nothing, even obvious issues | Baseline is passive-only by design — it never sends an attack payload, so a green baseline is not proof of a secure app, only of a quiet crawl | Dynamic Analysis in Practice |
| Checkov/tfsec passes Terraform you know is misconfigured | Wrong --directory (scanning a module folder, not the root), a leftover #checkov:skip/#tfsec:ignore comment, or scanning raw HCL where the risky value only exists after variable resolution in a real plan | IaC Hardening, Checkov, tfsec |
One #tfsec:ignore suppresses more than you intended | An inline ignore on a for_each/count block applies to every expanded instance, not the one copy you meant to except | tfsec |
| Findings you already fixed keep reappearing in the report | A Checkov baseline file frozen during onboarding, never regenerated — it's now hiding a fixed finding's fingerprint instead of a live one, or the fix landed on the wrong resource copy | Checkov |
| Policy-as-code blocks everything, including valid plans | The Rego rule tests only the resource type, not the actual risky attribute — e.g. matching every aws_security_group_rule regardless of what CIDR it actually opens | OPA & Conftest, IaC Security & Policy as Code |
| A Kyverno rule rejects a resource kind it was never meant to touch | match.any scoped too broadly, or the background controller is re-flagging a pre-existing resource on its scheduled scan | Kyverno |
A brand-new Enforce rule breaks a routine deploy that worked yesterday | The rule shipped straight to Enforce with no Audit-mode tuning period — pre-existing non-compliant resources were never surfaced first | Kyverno |
| Conftest/OPA reports failures in CI, but the merge isn't blocked | The pipeline never checks the tool's exit code, or the rule is named warn when it needed to be deny | OPA & Conftest |
| A Rego policy parses cleanly but never matches anything | Rego v1 dialect mismatch — if/contains syntax against an OPA build still defaulting to the pre-v1 dialect, or the reverse | OPA & Conftest |
| A policy written for Kubernetes YAML does nothing against a Dockerfile | Conftest hands a Dockerfile to Rego as an instruction array with no .kind field — a policy checking input.kind silently matches nothing, no parse error at all | OPA & Conftest |
| InSpec profile "passes" with suspiciously few controls run | A missing profile dependency in inspec.yml, or an overly narrow --controls filter left over from local debugging | Compliance as Code at Scale, InSpec |
| OpenSCAP reports zero applicable rules for the profile | Wrong --profile XCCDF ID passed, or the SCAP content bundle doesn't match the target's actual OS/version | OpenSCAP |
| DefectDojo shows the same CVE twice, as two separate findings | The same vulnerability was imported under two different Engagements instead of reused, so deduplication never gets a chance to run against it | Vulnerability Management & Triage, DefectDojo |
| A False Positive marking doesn't survive a re-import | The scan's parser is configured with a deduplication algorithm that doesn't produce a stable fingerprint across imports | Vulnerability Management & Triage |
| Prowler/ScoutSuite reports a clean account you know has issues | Wrong AWS profile or region scanned, or the IAM role lacks read permission for the service — an access-denied read failure looks identical to "no findings" | Cloud Security Posture, Prowler, ScoutSuite |
| A Falco rule is loaded but never fires | The eBPF probe/kernel module never actually attached, or the rule's condition is scoped too narrowly to match real syscall activity | Falco |
A few of the rows above hinge on a detail that has genuinely changed across tool versions — Rego's if/contains syntax became the OPA v1 default only in mid-2024; Kyverno moved validationFailureAction from a policy-level field toward a per-rule failureAction in newer releases; Checkov fails the build by default while Trivy's default exit code is 0 regardless of findings. Confirm the behavior of whatever version is actually pinned in the CDP's environment — --version, --help, and the tool's own error message when something doesn't parse are all faster and more reliable than remembering which release changed what.
When you're stuck
☺ Like you're 10: If you've been stuck a while, stop and check the boring things — are you even scanning the room you think you're in?
Every experienced practitioner has lost real minutes to a problem that didn't exist. These checks cost fifteen seconds and resolve a surprising share of "impossible" failures — and inside a 6-hour challenge window with a 24-hour report clock waiting behind it, fifteen seconds spent here is cheap insurance against losing twenty minutes later.
# 1. Am I actually looking at what the task described? pwd && git remote -v && git branch --show-current # 2. Does the config file I think is loaded actually exist, at that path? ls -la .semgrep.yml .checkov.yaml .gitleaks.toml 2>&1 # 3. Is the thing I'm scanning even reachable — target up, credentials valid? curl -sI "$URL" | head -1 aws sts get-caller-identity 2>&1 || echo "no valid AWS session"
- Re-read the task wording, verbatim. Candidates solve the problem they remember rather than the one on screen — scanning
stagingwhen the task namedprod, or gating on a severity the task never asked for. The wording usually names the exact repo, branch, image, or URL you're meant to use. - Confirm you're not still holding the previous challenge's context. A live environment carries state between tasks the same way a multi-cluster exam does — the previous challenge's target, token, or working directory is a common source of a scan that "should" work and doesn't.
- Time-box hard. Past five to seven minutes with no visible progress on one challenge, note where you are and move to the next one — five challenges in a 6-hour window rewards banked partial progress across all of them over a perfect solve on just one. See the CDP exam guide for how the two clocks actually work.
Practice routing, not fixing. On a disposable repo with a small app, container, and a couple of Terraform resources, break it five ways, one at a time: point a Semgrep --config at a path that doesn't exist; set a ZAP baseline scan running with no authentication context against a login-gated app; write an OPA rule that denies every aws_security_group_rule regardless of its CIDR; leave a Checkov --skip-check covering the exact rule you're supposed to be graded on; and set a Kyverno policy to Enforce against a namespace that already has three non-compliant Pods running. Run only the five-step opener each time, then say out loud which symptom-index row it is before you touch the fix. If you can name the row before the fix, the index is doing its job.
Practice the triage under real pressure
☺ Like you're 10: Reading the table once is not the same as finding the bug with a clock running. These drills are the clock.
The symptom index is a lookup table, not a substitute for having actually chased each of these down once. These timed drills put you in front of the real failure shape, not a description of it:
Leaked Credential Triage
A verified secret in CI — practice the rotate-first, clean-history-second sequence under a clock.
DrillVulnerable Dependency Fire Drill
A transitive CVE three layers down a dependency graph — where the "zero findings" and "same CVE keeps coming back" rows come from.
DrillWrite a Policy-as-Code Rule
Write a Rego rule narrow enough to catch the real risk without blocking every valid plan alongside it.
DrillFix a Broken Terraform Plan
A misconfiguration a scanner should have caught and didn't — practice the path and suppression checks first.
DrillContainer Escape Investigation
Runtime detection gone quiet — the Falco row above, worked end to end.
Benny the Beaver: SAST's clean, SCA's clean, container scan's clean — this challenge is basically done. Moving on.
Timmy: Clean, or did any of the three actually say how many files it looked at?
Benny: ...the SAST one says "0 files matched." Oh. The config path has a typo.
Foxy: Meanwhile my DAST run finished in ninety seconds and found three routes. Three. On an app with a whole dashboard.
Timmy: Did you give it something to log in with?
Foxy: ...no. It's been crawling the login page for ninety seconds and calling that a scan.
Recon: BEEP. And my policy just rejected a completely valid plan. Again. Fourth one today.
Professor Owl: Read the rule back to yourself, Recon — does it check the resource type, or the actual dangerous value on it?
Recon: ...just the type. Every security group rule, any CIDR. BEEP. Recalibrating.
Timmy: Three different tools, three different silences, and every single one had already told you why — before any of you tried to "fix" the code.
That's the whole page: five steps that take under two minutes, a table that turns whatever's on your screen into a lead, and the reminder underneath both of them that a quiet scanner is a symptom, not a result. Pair it with the CDP exam guide for how the two exam-day clocks actually run, the CDP study plan for the week-by-week route to exam-ready, Know It Cold and the command & tool reference for the from-memory drills this page assumes, and the capstone lab track for hands-on reps across the whole toolchain this index covers. Then close all of them and go find something to deliberately break.
1. Name the five steps of the universal triage order, in order. 2. A SAST scan reports zero findings. What's the most likely cause this page argues for, and what single line of output would confirm or rule it out? 3. A DAST scan finishes fast and reports only a handful of low-severity alerts. What's almost certainly missing, and what does a green report actually prove in that state? 4. A policy-as-code rule is rejecting every Terraform plan, including ones with no real risk in them. What's the most likely bug in the Rego, using the security-group example from this page? 5. Why does this page insist on re-running the exact same command after a fix, rather than trusting that the fix was correct?
Check your answers
- Read the tool's own self-report line → confirm the target actually scanned → confirm the ruleset/policy that loaded → confirm auth/session/credential state (for anything live) → only then form a hypothesis.
- Most likely a config or path problem, not a clean codebase — a
--configflag that resolves to nothing, or an ignore pattern covering the whole source tree. The self-report line itself ("N rules run on M files") confirms or rules it out immediately: ifMis 0, nothing was ever scanned. - An authentication context — the spider never got past the login shell, so it mapped three or four public routes and stopped. A green, low-alert report in that state proves the scanner crawled a small public surface cleanly; it proves nothing about the authenticated application behind the login, which is usually where the real attack surface lives.
- The rule almost certainly tests only the resource type (e.g.
rc.type == "aws_security_group_rule") without also checking the actual risky attribute, such as whether the rule'scidr_blocksactually includes0.0.0.0/0. Without that second condition, the rule denies every security group rule regardless of how safely it's configured. - Because a fix that isn't re-verified against the same command is a claim, not a proof — the same class of mistake a scanner's own silent config error can produce. Re-running the identical command confirms the fix changed the thing you thought it changed, rather than something adjacent to it.