The Detection Rules That Passed Every Test and Never Fired Once

Here's a dumb reason two of my detection rules never caught anything: I tested them against events that don't look anything like the events Sysmon actually sends.

I built a full Active Directory attack chain in an isolated lab (kerberoasting, targeted and full DCSync, a cross-domain privilege pivot, service-context shell spawns) and scored every phase against what my Wazuh SIEM actually produced. Most of it worked. Two cells came back blank, which is the normal, boring outcome of this kind of exercise: you find gaps, you close them, you move on. What I didn't expect was that two of my custom detection rules had passed wazuh-logtest, the vendor's own rule-testing tool, and had never once fired in production. Three separate bugs, each one hidden behind the one before it, all invisible to the tool whose entire job is to catch exactly this.

That's the part worth reading for. The blank cells were routine. The rules that lied to their own test suite were not.

A range, a canary, and a tripwire

The target is GOAD-Light, a two-domain Active Directory forest: a parent root domain and a child, the standard Game-of-Thrones-named hosts anyone who's touched GOAD will recognize. Nothing about the target side is interesting. It's a textbook enterprise AD, on purpose.

The blue side is where the work is: Wazuh as the SIEM and manager, Sysmon on every Windows host feeding the event channel to the Wazuh agent, Velociraptor for DFIR hunting, Suricata on a switch mirror port for the network view. The whole range sits on an air-gapped segment with a tripwire watching for any packet that tries to cross toward the management network. If the lab ever talks to something it shouldn't, I want to know before I find out the hard way.

Before scoring anything, I fire a known-good detection (an AS-REP roast that trips rule 100202) and confirm it lands. That's the canary. It proves the pipeline is alive end to end, which is what makes a blank cell mean something later. A VOID result after a live canary means the blue team was actually blind, not that the run silently failed.

The run: DETECT vs VOID

Here's the whole chain, scored. Counts are per-phase firings read straight out of alerts.json, not asserted.

Two of those rows earn a second look before moving on.

Sixty alerts become one

A full DCSync replication reads every object in the directory, and rule 100201 fires once per read. So a single DCSync doesn't produce one alert, it produces a burst of about sixty. That's not a detection, that's a denial-of-service on whoever's on call. Rule 100203 chains off 100201 with frequency=8, timeframe=30, same_field win.eventdata.subjectUserName: eight-plus replication reads from one non-machine principal inside thirty seconds collapse into a single alert, attributed to that principal. In this run the full replication went 59 to 1. The cross-domain pivot went 53 to 1 on the root DC. Burst becomes signal, and the analyst reads one line instead of sixty.

The honest warning sign

The 972 is the row I'm not going to dress up. Rule 100300 fires when a service-context process spawns a shell (a svchost or Program-Files parent) and it did catch the real service-to-shell spawn in this chain. It also caught 972 benign firings from the lab's own activity bot, whose svchost-hosted scheduled tasks run cmd.exe → powershell roughly once a minute. A true positive you can't find in the noise isn't a detection you can rely on. It's a rule that technically fired and practically didn't help anyone.

Three bugs, each hiding behind the last

Before this run, rules 100300 and 100301 had a green light. They passed wazuh-logtest. The live run proved they had never fired in production, not once. Peeling that apart took three fixes, and each one only became visible after the last was fixed.

First, field-name case. The rules matched win.eventdata.Image, ParentImage, ImageLoaded, capitalized, the way they appear in Sysmon's own XML. But the Windows event-channel decoder lowercases the first letter, so the real keys are image, parentImage, imageLoaded. Every built-in Sysmon rule already knows this: grep them and you get 76 uses of image, 36 of parentImage, 7 of imageLoaded, and zero capitalized. My logtest fixture used the capital keys, so the gate happily validated a fixture that didn't match reality.

Second, no base-rule chaining. Fix the casing and the rules still don't fire. A custom rule that matches win.system.eventID directly is never evaluated on the event-channel path at all. Sysmon events arrive under a decoder tree, and a rule has to chain into it the way the built-ins do: <if_group>sysmon_event1</if_group> for EID-1 off base rule 61603, <if_group>sysmon_event7</if_group> for EID-7 off base rule 61609. Match the raw event ID standalone and the rule is simply never reached, no error, no warning.

Third, doubled backslashes. Casing fixed, chaining fixed, and one branch still misses. The agent serializes event-data paths with doubled backslashes: C:\Program Files\… arrives over the wire as C:\\Program Files\\…. An anchored single-backslash literal like ^C:\Program Files\ never matches that. The svchost branch of the same rule worked the whole time, purely by accident, because its leading .* absorbed the extra backslash and hid the bug from me. The fix is \\+ (one or more backslashes) in every anchored path literal.

The meta-lesson is the one I actually want people to walk away with: wazuh-logtest cannot test event-channel rules at all. Pipe an event into it and it decodes as generic syslog, rule 1, and never reaches the Sysmon tree, so your if_group chain is never exercised. The gate I trusted was validating a hand-built fixture, not production. The only valid test is a live purple-test that reads alerts.json after the fact.

That's three deploy cycles I paid for so you don't have to.

Closing the VOID cells

The two blank cells were the actual point of the exercise, so they didn't get to stay findings.

EID-7 was a collection gap, confirmed at the source. On the host, Sysmon was generating 0× EID-7 against 570× EID-1, a generation gap, not a forwarding gap. The cause was a trap with three layers of its own: Sysmon's running config lived only in the registry, because the on-disk C:\Windows\Temp\sysmonconfig.xml had been deleted; its ImageLoad include-group was present but empty, so zero rules meant zero events; and Sysmon64.exe -c replaces the config wholesale, so a naive "just add EID-7" file would have wiped the twenty-plus SwiftOnSecurity groups feeding everything else. The fix was to take the lab's version-controlled base config, add exactly one Program-Files-scoped user32.dll rule to its existing ImageLoad group, and apply the complete file. Rule 100301 went from a lifetime total of zero firings to firing on real events across all three hosts: WINTERFELL twice, CASTELBLACK once, KINGSLANDING once.

The noisy 100300 got made actionable instead of gutted. Rather than weaken the rule, I added a scoped allowlist (rules 100303 and 100304, level 0) that suppresses the known-benign service-context spawns: the activity bot's bot_rdp.ps1 and New-PSDrive, the Wazuh agent's own compliance checks, a handful of Windows maintenance tasks. I verified it two-sided: the bot's alerts dropped to zero while the bot kept firing the underlying EID-1 the whole time, and a genuine, non-benign svchost → cmd spawn still fired at level 11. No over-suppression. The shells spawned by my own Velociraptor were deliberately left firing: remote-exec through your DFIR tool is exactly the abuse you want a SIEM to flag.

Both fixes are baked into a fresh lab snapshot, so they survive a reset instead of evaporating on the next rebuild.

Nothing escaped

The part that matters regardless of how the detections went: at teardown the tripwire had logged six frames, every one an expected management-plane control ping, zero lab-to-external leak. The chain was read-only the whole way through (no directory objects were mutated), so there was nothing to roll back. Whatever ran in that segment stayed in that segment.

What this is actually worth

None of the individual bugs here are novel. Sysmon's lowercase-first field names, the need to chain off if_group, the doubled-backslash serialization. Someone, somewhere, has hit each of these before. What I don't think is common knowledge is that all three can stack silently behind a green logtest result, and that logtest itself can't see event-channel rules at all. That's not a corner case. That's the default failure mode for anyone who trusts the vendor's own gate the way I did.

The value here isn't a discovery. It's a repro: exact rule IDs, exact counts, exact fixes, and a method. Canary first, live purple-test, read alerts.json, never trust a fixture you built yourself to validate a rule you also built yourself.

If you run Wazuh with Sysmon, take these

• Event-channel event-data keys are lowercase-first. image, not Image.
• Custom Sysmon rules have to chain via if_group. Matching win.system.eventID standalone silently never fires, with no error to tell you.
• Event-data path values carry doubled backslashes. Use \\+ in anchored literals, or lead with .*; never an anchored single backslash before a path segment.
• wazuh-logtest cannot test event-channel rules. They decode as syslog and skip the Sysmon tree entirely. Validate with a live purple-test and read alerts.json, not the gate's exit code.

Green cells feel good. The blank ones, and the ones that only look green, are where the detections you don't actually have are hiding.

Run in a sealed, air-gapped lab on infrastructure I own. The attack chain was read-only: nothing was written to any external system, and nothing left the segment. If you reproduce this, contain it the same way.