Detection Engineering

TheoryHard15 minDetection EngineeringSIEM Operation

Precision vs Recall: The False-Positive / False-Negative Tradeoff

What is it?

Every detection lives on a tradeoff. A true positive is a real attack the rule caught; a false positive is benign activity it flagged; a false negative is a real attack it missed; a true negative is benign activity it correctly ignored. Precision is how many of the alerts are real; recall is how many of the real attacks were caught. Widening a rule raises recall but usually lowers precision, and vice versa — tuning is choosing the right point on that curve.

Why it matters

Both failure modes have real costs. Too many false positives cause alert fatigue: analysts stop trusting the rule, mute it, and then it catches nothing — a false positive problem becomes a false negative problem. Too many false negatives mean real attacks sail through. A detection engineer who cannot reason about this tradeoff either floods the SOC or blinds it.

Where you see it

The tradeoff shows up as alert volume and quality in the SIEM: a rule firing hundreds of mostly-benign alerts a day has poor precision; a rule that never fires during a known intrusion has poor recall. It is measured by triaging a sample of alerts (how many were real) and by testing against known attacks (how many were caught).

What normal looks like

A well-tuned detection fires rarely and is right most of the time, while still catching the attack it was built for. It uses context — thresholds, allowlists, corroborating conditions — to keep precision high without sacrificing the recall that matters. Analysts act on its alerts because experience has taught them to.

What suspicious looks like

A broken detection sits at an extreme. The over-broad rule (alert on all PowerShell) has near-perfect recall and useless precision — a wall of noise. The over-narrow rule (one exact command line) has high precision and terrible recall — it misses any small variation. Both look 'deployed' but neither protects anyone.

How analysts investigate

Estimate both sides before deploying: run the rule over historical data to see how many alerts it would have produced and how many would be benign (precision), and test it against known attack samples and variants to see how many it catches (recall). Then tune toward balance with context — narrow the selection, add allowlists, require a corroborating condition — and re-measure.

Common beginner mistakes

Chasing 100% recall with an over-broad rule and drowning the SOC in false positives.
Chasing 100% precision with an over-narrow rule that misses every variant.
Ignoring alert fatigue — a rule so noisy it gets muted detects nothing at all.

What you will be able to do

Define true/false positive/negative, precision and recall.
Diagnose whether a rule is too broad or too narrow.
Tune toward balance with thresholds, allowlists and corroboration.

Every rule makes two kinds of mistake, and you cannot usually eliminate both at once. Miss real attacks (false negatives) or flag benign activity (false positives) — tuning is the art of trading one against the other until the rule is useful. The trap is thinking either extreme is safe: a rule that catches everything is as worthless as one that catches nothing, because a wall of false positives gets muted and then catches nothing too.

The rule fired	It was an attack	It was benign
Yes (alert)	True positive ✓	False positive (noise)
No (silent)	False negative (missed!)	True negative ✓

Worked example — two broken rules

# Rule X — too broad
detection: { child_image: '*\powershell.exe' }   # ANY powershell
# result: ~800 alerts/day, ~5 real -> precision terrible

# Rule Y — too narrow
detection: { command_line: 'powershell -enc SQBFAFgA' }  # one exact string
# result: 0 alerts on the next build -> recall terrible

Both rules are 'deployed'. Before reading on, decide: what is wrong with each, and how would you fix them into one good rule?

The answer: Rule X has near-perfect recall but useless precision — PowerShell runs constantly for legitimate reasons, so 800 daily alerts bury the 5 real ones and the rule gets muted. Rule Y has high precision but terrible recall — it matches one exact encoded string, so any new build slips past. The fix is neither extreme: target the behaviour from Lesson 1 (a document app spawning an encoded PowerShell child), then tune with context — require the suspicious parent, add an allowlist for the known admin automation host, and perhaps a threshold. That single rule has both good precision and the recall that matters.

Recap

Precision = how many alerts are real; recall = how many real attacks are caught.
Too broad = false-positive flood; too narrow = false-negative misses; tune to balance.
Alert fatigue is real: a muted noisy rule detects nothing — precision protects recall.