When ML fails: the zero-visibility problem

When a firewall blocks something, it leaves a trace. When an antivirus quarantines a file, someone gets a notification. But when a detection model decides that a malicious action is harmless, nothing happens at all. There is no alert to investigate, no log entry marked “suspicious”, no ticket in the security operations centre (SOC). The attack simply continues, and the defenders do not know there is anything to look for. Security teams sometimes call this zero visibility.

Zero visibility is not unique to machine learning; any detection method can miss. What changes with ML is that teams can come to trust a single model’s silence as proof that nothing is wrong. If that model is the only thing watching, a successful evasion turns into free movement for the attacker in the phase that follows: exploring the network, collecting data and preparing to take it out.

Design for the miss, not just the hit

The practical response is a mindset often called assume breach: plan as if one of your defences will fail, and make sure the next one would notice. NIST’s incident response guidance (SP 800-61) and its continuous monitoring guidance (SP 800-137) both build on this idea of multiple, overlapping sources of evidence rather than a single gatekeeper.

Tripwires that do not depend on a model

Some of the most reliable signals are deliberately simple. A canary token is a fake credential, file or link placed where no legitimate person should use it; the moment anything touches it, you know. MITRE’s D3FEND knowledge base catalogues these decoy techniques alongside other defensive measures. Because they do not rely on classification, an attacker who fooled the model can still trip them.

Keep the evidence even when nothing is flagged

• Log more than alerts: keep connection records, process starts and logins long enough to look back weeks later.
• Record what the model decided and why, including “benign” verdicts, so a later investigation can see what it waved through.
• Watch outbound traffic as closely as inbound: data theft needs to leave, and it has to go somewhere.
• Test your coverage: an authorised exercise that checks whether each ATT&CK technique relevant to you would leave a trace somewhere.
• Review the “normal”: schedule time to look at what automated tools consider routine.

Where FireAI fits

On a Mac, FireAI provides one of those independent layers. It records which app connected where, including connections it allowed, and draws them on the world map, so a program that passed every other check still leaves a visible trail when it starts talking to a new country or company. Connections are kept in its activity history for later review, and Investigate lets a person look at the facts behind any connection, not only the ones that were flagged.