Detection Engineering & Security Observability

Most teams that say they "do detection" really mean they turned on whatever came bundled with their SIEM license. That's not nothing — but a vendor's default rule pack was tuned against nobody's environment in particular, and it has no idea which of your services actually holds the data an attacker would want. Detection engineering treats detections the way this whole course treats everything else security-related: as code you write, version, test, and deliberately improve, aimed at your own environment and informed by how real adversaries actually behave. This page covers the three pieces that make that discipline concrete — MITRE ATT&CK as a map of adversary behavior you can measure your coverage against, Sigma as the language you write portable, testable detection rules in, and the SIEM/XDR pipeline — Wazuh is one open-source implementation this course covers in depth — that turns a raw log line into an alert worth a human's attention instead of one more thing they've trained themselves to click past.

Turning on the default rule pack isn't a detection program

☺ Like you're 10: A rule that only recognizes one attacker's exact fingerprint stops working the moment that attacker washes their hands. A rule that recognizes the way they picked the lock keeps working no matter how many times they change gloves.

SAST, DAST, and SCA catch flaws in what you built and what you pulled in before any of it ships. Detection engineering is the same shift-left discipline pointed at what happens after something ships and runs — except most organizations never actually build it as a discipline. They buy a SIEM or XDR platform, accept the hundreds or thousands of built-in correlation rules it arrives with, enable them all, and call the box checked. The rules were tuned by the vendor against a generic model of "an enterprise," not against your actual log sources, your actual service topology, or the specific things an attacker would want from your specific environment — so a large share of them fire on noise unique to your stack, and an equally large share stay silent on the exact technique an attacker would actually use against you.

David Bianco's Pyramid of Pain (2013) explains precisely why the distinction matters, not just that it does. It ranks indicator types by how expensive they are for an attacker to change once you start detecting them:

A vendor's default pack skews toward the bottom of this pyramid, because hash and IP matching is cheap to ship generically — it needs no understanding of your environment at all. Building rules at the top of the pyramid, keyed on a technique rather than a fingerprint, is exactly what the rest of this page is about.

MITRE ATT&CK as a coverage map, not a checklist

☺ Like you're 10: A shelf of "we have an alarm system" badges tells you nothing about which doors it actually watches. A floor plan colored in — green where a sensor is armed and tested, gray where nobody's checked — tells you exactly where to worry.

MITRE ATT&CK is a public, continuously updated knowledge base of real adversary behavior, built from incident reports, malware analysis, and red-team operations rather than invented top-down. Its Enterprise matrix is organized into three layers of increasing specificity: a tactic is the "why" — one of fourteen adversary goals such as Initial Access, Privilege Escalation, Credential Access, or Exfiltration; a technique is the "how" — a specific method of achieving that goal, identified by an ID like T1059 (Command and Scripting Interpreter); and a sub-technique narrows that further, like T1059.001 specifically for PowerShell. MITRE ships a new version of the matrix roughly twice a year, so treat any specific version number you read, including anything on this page, as a snapshot — check attack.mitre.org for what's current.

Turning the matrix into a heatmap with the Navigator

Read on its own, ATT&CK is a glossary — useful for naming things precisely, but not yet a management tool. The ATT&CK Navigator (a browser-based tool MITRE also publishes) turns it into one: you build a "layer," a JSON file scoring every technique cell your team cares about, and the Navigator renders that as a color-coded heatmap over the matrix. A common scoring convention runs 0 through 3 — 0 no visibility into this technique at all, 1 a relevant data source exists but nothing alerts on it, 2 a rule exists but hasn't been proven to fire, 3 a rule exists and has been tested against real or simulated attacker behavior. That last distinction is the entire point of this exercise: "we have EDR, so we're covered" collapses the moment you separate "a data source exists" from "a tested rule actually watches it," and a Navigator layer forces that separation onto the page instead of leaving it as an assumption nobody checked.

Two open-source tools make building that layer file less of a manual chore. DeTT&CT, open-sourced by the cyber defence centre at Rabobank, lets you describe your data sources and your detection rules in YAML and generates ATT&CK Navigator layers automatically — including a separate "data quality" score, because a log source that exists but is missing the field a rule actually needs to key on isn't real visibility either. And because threat modeling already asks "what would an attacker want and how would they get it" at design time, ATT&CK is the same question's answer key, built from what real adversaries actually did across thousands of documented intrusions — pairing the two turns a threat model's guesses into a checkable, citable catalogue.

Anatomy of a Sigma rule

☺ Like you're 10: One recipe card, written in plain language, that a dozen different kitchens can each translate into their own cooking style — you write the logic once, and the specific SIEM figures out how to actually run it.

Sigma is a generic, YAML-based detection-rule format created by Florian Roth and Thomas Patzke, maintained today under the SigmaHQ GitHub organization. Its job is to let you describe detection logic once — "flag process X invoking behavior Y" — without hardcoding the query syntax of any particular SIEM. A separate converter compiles that generic YAML into whatever query language your backend actually speaks: Splunk SPL, Elastic Query DSL or EQL, Microsoft Sentinel KQL, and others. That decoupling is the whole value proposition — the rule is portable, diffable, and reviewable in a pull request exactly the way this course already argued application code and infrastructure-as-code should be.

# A Sigma rule is generic YAML — it doesn't know or care which SIEM will eventually run it
title: Suspicious PowerShell Encoded Command Execution
id: 8c8c8b1e-1a2f-4e3d-9b7a-3f2c9d1e0001
status: experimental
description: >
  Flags powershell.exe invoked with an encoded/obfuscated command,
  a common way to hide malicious commands from casual log review.
references:
  - https://attack.mitre.org/techniques/T1059/001/
author: Shift-Left Squad
date: 2026-01-15
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    Image|endswith: '\powershell.exe'
    CommandLine|contains:
      - '-EncodedCommand'
      - '-enc '
      - ' -e '
  filter_sccm:
    ParentImage|endswith: '\ccmexec.exe'
  condition: selection and not filter_sccm
falsepositives:
  - Legitimate admin scripts that pass encoded commands
  - Configuration-management tooling — adjust filter_sccm for your own inventory
level: medium
tags:
  - attack.execution
  - attack.t1059.001
  - attack.defense_evasion
  - attack.t1027

The logsource block: telling Sigma what data it needs

logsource is an abstraction, not a literal field name — category: process_creation plus product: windows describes the kind of event the rule needs, and it's the converter's job to map that abstraction onto whatever your actual pipeline calls it: Sysmon Event ID 1, a Windows Security 4688 event, or an EDR vendor's own process-start schema. When your environment's field names don't match Sigma's default taxonomy, pySigma lets you supply a processing pipeline — a YAML mapping file that translates Sigma's generic field names (Image, CommandLine) into your actual log's field names before compiling the query. Skipping this step is the single most common reason an imported Sigma rule silently never fires: it compiles without error and simply never matches anything in your schema.

The detection block and condition logic

The detection block holds one or more named selectors — each a map of field-to-value matches — plus a condition expression that combines them with boolean logic. Field modifiers after a pipe (|endswith, |contains, |re, |cidr, |startswith) control how the match is made; a list of values under one field is implicitly OR'd together. Above, selection defines what counts as suspicious and filter_sccm carves out one known-legitimate source, and condition: selection and not filter_sccm is the rule's actual logic — the two named blocks are just its building blocks. This structure is why a Sigma rule reads like a small, testable unit of logic rather than a giant regular expression: you can reason about selection and filter_sccm independently, and a reviewer can suggest a second filter without touching the core match at all.

Compiling one rule into many backends

The modern tooling is pySigma, a Python library, and sigma-cli, the command-line front end built on it — together they replaced the older sigmac converter around 2023. Each target SIEM is a separate installable backend package.

# one rule, compiled to whichever query language your SIEM actually speaks
pip install sigma-cli pysigma-backend-splunk pysigma-backend-elasticsearch

sigma convert -t splunk -p sysmon encoded_powershell.yml
#   index=win_events Image="*\\powershell.exe" CommandLine="*-EncodedCommand*" ...

sigma convert -t elasticsearch -p ecs_windows encoded_powershell.yml
#   process.executable:*\\powershell.exe AND process.command_line:*-EncodedCommand* ...

sigma check encoded_powershell.yml       # lints structure before it ever reaches a SIEM

Correlation: rules that span more than one event

☺ Like you're 10: One knock on the door isn't suspicious. Twenty knocks in a minute, followed by the door actually opening, is a completely different story — and you can only tell the story by counting, not by looking at any single knock.

Most real attacker behavior isn't visible in one log line — a brute-force attempt is a pattern of many failed logons, and a compromised account behaving oddly is a pattern of ordinary-looking single events happening in an unusual sequence or volume. A single-event Sigma rule is structurally blind to this, because it evaluates each event on its own. The Sigma specification addresses this with correlation rules, which reference one or more base rules by ID and add a temporal or counting condition across the matches those base rules produce, grouped by a shared field. The exact set of correlation types (event_count, value_count, temporal, and temporal_ordered among them) and their required keys are still an area the Sigma specification is actively refining, so treat the shape below as illustrative and check SigmaHQ's current specification before shipping it.

# references base rules by id — this rule only fires on a PATTERN across events,
# not on any single event by itself
title: Brute Force Followed by Success From the Same Source
id: 3f7a2b10-9c4d-4e5f-8a1b-7d21e4c50002
status: experimental
correlation:
  type: temporal_ordered
  rules:
    - failed_logon      # a base Sigma rule matching a failed-logon event
    - successful_logon   # a base Sigma rule matching a successful-logon event
  group-by:
    - SourceIp
  timespan: 10m
  condition:
    gte: 5               # at least 5 failed_logon matches before the success
level: high
tags:
  - attack.credential_access
  - attack.t1110

Correlation is also where Wazuh's own native rule engine and Sigma converge conceptually even though they're separate technologies: Wazuh rules support a frequency/timeframe mechanism to require N matches of a lower-level rule within a window before a higher-severity composite rule fires — the same "count and combine" idea Sigma's correlation type expresses in portable YAML, just written directly in Wazuh's own XML rule syntax instead.

From raw log to alert: inside a SIEM/XDR pipeline

☺ Like you're 10: A single word yelled across a noisy stadium means nothing on its own. It only becomes useful once someone collects every yell, translates them all into the same language, checks who's yelling and whether they're known troublemakers, notices the pattern in the yelling, and then taps exactly one person on the shoulder to go look.

A Sigma rule is logic. Getting from a raw log line on a hundred different machines to one alert a specific human investigates requires an entire pipeline around that logic, and every stage in it can silently degrade the signal before a rule ever gets to evaluate anything.

Six stages between "log line" and "page a human"

Collection gathers events from agents, syslog forwarders, and cloud audit-log APIs. Normalization reshapes wildly different raw formats — a Windows Sysmon event, a Linux auditd record, a cloud provider's JSON audit log — into a common field schema, using something like Elastic's ECS or the newer, vendor-neutral Open Cybersecurity Schema Framework (OCSF), which AWS, Splunk, and a group of other vendors launched in 2022 specifically so tools from different vendors could exchange normalized events without a bespoke mapping for each pair. Enrichment attaches context the raw event doesn't carry on its own — a threat-intel feed flagging a destination IP as known-bad, or asset and identity context from a CMDB or directory service saying "this account belongs to a service, not a human, so it should never see interactive logon at 2am." Correlation is where Sigma-derived and native rules actually evaluate the normalized, enriched stream — both single-event rules and the multi-event correlation rules from the previous section. Alerting deduplicates repeated hits, assigns a severity, and routes the result to wherever a human actually looks — a case-management queue, a SOAR playbook, an on-call page.

Wazuh as one open-source implementation of this pipeline

Wazuh is a concrete, freely available walk through every one of the middle five stages. A lightweight agent installed on each monitored host handles collection — shipping log data, file-integrity events, and configuration-assessment results back to a central point. The Wazuh manager applies decoders to parse raw log lines into structured fields (normalization), then evaluates its own XML-based rules against those decoded fields to correlate and assign each match a severity level from 0 to 15 — and can trigger an active response script automatically, such as blocking an IP after repeated failed logons, which is the automated-action layer a dedicated SOAR product would otherwise provide. The Wazuh indexer, built on OpenSearch, stores the resulting alerts, and the Wazuh dashboard is where a Tier 1 analyst actually triages them. One honest caveat: Wazuh's rule and decoder format is its own XML syntax that predates Sigma and isn't a native Sigma compilation target the way Splunk or Elastic are — porting Sigma-authored logic into Wazuh today generally means either hand-translating the rule into Wazuh's XML syntax or reaching for one of the community conversion projects, and tooling support for less-common backends changes quickly enough that it's worth checking the current state before you commit to a specific bridge.

One layer further down the stack, Falco plays a structurally similar role at the kernel level: its own YAML rule language matches syscall patterns in real time inside a running container, feeding a comparable collect-normalize-correlate-alert pipeline purpose-built for runtime behavior rather than aggregated logs. See container runtime security for that layer specifically.

Testing detections the way you test code

☺ Like you're 10: A fire drill you've never actually run is just a hope written on a piece of paper. Running one — safely, on purpose — is the only way to find out if the alarm actually rings.

An untested detection rule is a guess dressed up as a control. It compiled without error and it reads as though it should fire — neither of those facts tells you whether it actually will, against a real invocation of the technique it claims to cover, on your actual log pipeline. Atomic Red Team, an open-source library maintained by Red Canary, closes that gap directly: it's a collection of small, self-contained "atomic tests," each one implementing exactly one ATT&CK technique or sub-technique in isolation — a specific LSASS-dumping command for T1003.001, a specific encoded PowerShell invocation for T1059.001 — runnable through the companion Invoke-AtomicRedTeam module or an equivalent client. The workflow this enables is the direct, practical answer to the coverage-map problem from earlier on this page: pick a gray or orange cell in your Navigator layer, run the matching atomic test in a controlled lab, and watch whether your Sigma-derived rule actually fires. If it does, the cell earns a 3, with a dated test run to back the claim. If it doesn't, you've found a real gap before an actual attacker did, instead of after.

This is also where purple teaming earns its name: instead of a traditional red-team engagement that withholds findings until a report weeks later, the offensive operator and the detection engineer work the same session together in real time — the red side executes a technique, the blue side watches live and reports exactly what did or didn't fire, and tuning happens on the spot instead of in a post-mortem. See offensive security for DevSecOps for the attacker-side tooling this pairs with, and the OSCP for one place those offensive techniques get formalized if you want to go deeper than this page does. Running these tests as a one-off exercise before an audit is better than nothing, but it's still a snapshot — security chaos engineering covers what it looks like to run this kind of controlled attack simulation continuously and automatically rather than annually.

The base-rate problem: why "it worked in the lab" still floods the SOC

☺ Like you're 10: Even a lie detector that's right 99 times out of 100 will accuse mostly-innocent people, if you test a million mostly-innocent people and only one of them is actually lying.

A rule can pass every atomic test in the previous section and still be operationally useless, for a reason that has nothing to do with the rule's logic and everything to do with arithmetic. Stefan Axelsson's 2000 paper "The Base-Rate Fallacy and the Difficulty of Intrusion Detection" laid this out precisely: when the thing you're looking for is extremely rare relative to the total volume of events you're scanning, even a very low false-positive rate produces a very high false-positive count, because that rate multiplies against an enormous population of benign events while true positives multiply against almost nothing.

Work the numbers concretely. Say a rule has a 1% false-positive rate — sounds tight — and your environment produces a million relevant events a day, of which real attacks account for roughly one. The rule generates about ten thousand false alerts for every one true attack it might catch. No SOC survives that ratio; analysts either drown or, more likely, start reflexively dismissing the rule's alerts entirely — which is functionally identical to having no rule at all, except now it also cost engineering time to build. This is precisely why the earlier stages of the pipeline matter as much as the rule itself: enrichment narrows a raw match down to one corroborated by asset or identity context before it counts as suspicious, and correlation requires a pattern across multiple weak signals instead of paging on any single one — both exist specifically to push the effective false-positive rate down far enough that the base-rate math stops working against you.

Operationally, this is why mature detection programs tier their response instead of paging on every single hit: a Tier 1 analyst triages the initial alert against runbooks and dismisses or escalates it in minutes; a Tier 2 analyst investigates anything escalated, pulling in the kind of timeline reconstruction covered in incident response and forensics; and detection engineers — Tier 3, in effect — feed every confirmed false positive back into tuning the rule rather than treating the noise as an acceptable cost of doing business. Mean-time-to-detect and mean-time-to-respond are the metrics that make this loop's health visible over time, the same way compliance and governance already covers tracking metrics as evidence rather than as vanity numbers.

Detection-as-code: rules live in git, not just the SIEM UI

☺ Like you're 10: A rule someone typed straight into a settings screen last Tuesday, with no record of who changed what or why, isn't something you can trust — or defend — six months later.

Security as code — reviewable, testable, versioned rules instead of a person's memory — is the thread this whole course pulls on, and detection rules are exactly as much "security as code" as an OPA policy or a SAST rule set. A Sigma rule edited directly in a SIEM's web console has no pull request, no reviewer, and no test run behind it; a Sigma rule stored in a git repository does, and that difference is the entire point of a detection-as-code pipeline.

# .ci/detections.yml — rules are code: linted, tested, and shipped silent before they page anyone
stages:
  - lint
  - test
  - shadow-deploy
  - promote

sigma-lint:
  stage: lint
  script:
    - sigma check rules/**/*.yml                                   # structural validation
    - sigma convert -t opensearch -p ecs rules/**/*.yml --dry-run   # confirms it actually compiles

sigma-test:
  stage: test
  script:
    # replay canned log fixtures through the rule and assert expected hits / non-hits
    - pytest tests/test_detections.py --sigma-rules rules/
    # then prove it against simulated attacker behavior in the lab range
    - invoke-atomictest T1059.001 --log-path ./lab-logs/
    - python verify_alert_fired.py --technique T1059.001 --window 5m

shadow-deploy:
  stage: shadow-deploy
  script:
    - deploy-rule.sh --mode silent rules/encoded_powershell.yml   # logs, never pages, for two weeks
  only:
    - main

promote-to-enforced:
  stage: promote
  when: manual   # a human reviews the shadow-mode false-positive rate before it can page anyone
  script:
    - deploy-rule.sh --mode enforced rules/encoded_powershell.yml

The staged rollout in that last block matters as much as the CI checks above it. A brand-new rule pushed straight into "enforced, paging" mode is a bet that your test fixtures caught every legitimate pattern your messy, real production environment will actually throw at it — a bet you lose more often than a shadow-mode bake-in period costs you in delay. Running silently first and reviewing the false-positive rate before promotion is the same "fail closed, but prove it's safe first" instinct that governs security in CI/CD gates elsewhere in the pipeline, just applied to the last line of defense instead of the first. See secure SDLC gates and the DevSecOps Maturity Model and the CDP toolchain for the CI mechanics this reuses.