Most organisations running a SIEM assume they detect attacks. Very few can say which ones.
Until recently that was an assertion from consulting practice. There are now figures from production environments, and they are less comfortable than the assertion.
The numbers that describe the problem
For its annual report on SIEM detection risk, CardinalOps analyses real configurations — most recently 2.5 million log sources, over 23,000 distinct source types and more than 13,000 detection rules across hundreds of production environments running Splunk, Microsoft Sentinel, QRadar, CrowdStrike LogScale and Google SecOps. The benchmark is MITRE ATT&CK.
| Finding | Value |
|---|
| ATT&CK techniques with a detection rule in place | 21 % |
| Techniques that could be covered with telemetry already ingested | over 90 % |
| Existing SIEM rules that are broken and will never fire | 13 % |
The middle figure is the decisive one. The data is not missing. The detection logic is. The telemetry is already in the SIEM; it simply is not being used. Answering the coverage gap with "we need to onboard more sources" solves a problem you do not have — and pays ingest costs for the privilege.
A note on the original: it cited a dwell-time figure from a Mandiant report. I have removed it, because it could not be attributed unambiguously to the edition named. The figures above carry the argument better anyway.
Three layers to the gap
First: no rule. For four techniques in five, no detection exists at all. That is the gap every heatmap shows.
Second: broken rule. Thirteen percent of existing rules never fire — misconfigured data sources, referenced fields absent from the current schema, logic errors. This is the more dangerous category, because it looks like coverage. The cell in the heatmap is green. Nothing happens when it matters.
The sentence that belongs in every board paper: rule count and functioning-rule count are two different numbers, and most organisations do not know the difference in their own environment.
Third: drift. Even rules that worked break — through schema changes, new log formats, a vendor update, a relocated data source. Coverage is therefore not a state you reach but one that decays without maintenance.
Why the heatmap alone misleads
ATT&CK Enterprise covers 14 tactics with roughly 200 techniques and more than 450 sub-techniques. A heatmap that is green at technique level can hide substantial gaps at sub-technique level.
And more green is not automatically more security. A coverage percentage weights every technique equally, when the techniques relevant to your sector and your adversaries are a fraction of the matrix. Forty percent coverage of the right techniques beats seventy percent of the wrong ones.
The measurement only becomes useful with three filters: relevance (which actors target our sector?), health (does the rule fire at all?) and data availability (does the log the rule depends on actually arrive?).
Where the gaps typically sit
- Persistence: registry modification, scheduled tasks, DLL hijacking — unmonitored in many default configurations.
- Lateral movement: pass-the-hash, RDP hijacking, WMI-based movement. One of the largest gaps there is.
- Identity-based attacks: token theft, session hijacking, OAuth app registration — attacks that leave no malware and that endpoint detection structurally cannot see.
- Cloud-native attacks: Entra ID manipulation, AWS IAM escalation, GCP service-account abuse. On-premise logs arrive reliably; cloud telemetry patchily.
- Non-human identities: service accounts, API keys, bot accounts. They now outnumber human users and are monitored less closely.
- Living off the land: PowerShell, WMI, PsExec. Indistinguishable from legitimate administration without a baseline.
- Exfiltration: egress over HTTPS, DNS tunnelling, cloud storage.
Detection engineering as a process
Detection rules are software. They are developed, tested, versioned and maintained — or they decay.
- Threat intelligence: which techniques do the actors relevant to our sector use?
- Data source mapping: which logs does the detection need, and do they actually arrive?
- Rule development: behavioural rather than signature-based.
- Testing: validate against real simulations — Atomic Red Team, MITRE Caldera.
- Tuning: reduce false positives without sacrificing detection.
- Measurement: document coverage and health against the matrix.
Step six is the one usually missing — and without it the other five cannot be verified.
Measuring MTTD properly
Mean time to detect is the central metric, but only per technique. A SIEM can detect brute force in seconds and remain permanently blind to lateral movement over WMI. The average conceals exactly the gap you are looking for.
MTTD becomes meaningful through purple-team exercises: execute defined ATT&CK techniques, measure whether and how fast they are detected, feed the result into the roadmap. Quarterly is a sufficient cadence.
What to do
- Run a coverage assessment against ATT&CK. Map the existing rules. DeTT&CT automates part of it.
- Check rule health, not just rule existence. For each rule: does the data source arrive? Do the referenced fields exist in the current schema? When did it last fire? A rule that has not fired in twelve months is either redundant or broken — you need to know which.
- Inventory data sources. What is actually collected, what is missing. But only after steps one and two: the odds are high that the data is already there.
- Detection as code. Rules in Git, tests in the pipeline, automated deployment. That is the only effective answer to drift.
- Set a purple-team cadence. Quarterly at minimum, with documented results.
- Prioritise by relevance, not by percentage. The techniques used by the actors targeting your sector come first.
From practice
Detection coverage is the foundation of any SIEM investment — and in most organisations it is never measured.
For us it is operational standard: the XIEM® stack is measured against the ATT&CK matrix and continuously extended. For each customer we document which techniques are covered, which data sources are missing and where the next rules will be built.
The element that pays off most clearly is rule health. A rule that exists is not yet a detection. So we check regularly whether each rule can technically fire at all — and we regularly find something.
The difference between a SIEM that collects logs and one that detects attacks sits exactly here.
Do you know what your SIEM is missing? We will show you — with an ATT&CK coverage assessment.
Sources