Network Security Monitoring (NSM): Data, Tools, Detection

At its core, network security monitoring (NSM) is the practice of collecting, analyzing, and escalating network data, the indications and warnings that reveal intrusions, so defenders can detect and respond before damage lands. Richard Bejtlich, who codified the discipline in The Practice of Network Security Monitoring, framed it as finding intruders on the network and acting before they hurt the enterprise.

Why gather all that traffic in the first place? Dwell time answers the question: Mandiant’s M-Trends 2026 analysis of frontline investigations puts global median dwell time at 14 days, up from 11 the year before. Organizations detected 52% of intrusions themselves, up from 43% the year before. The rest came from outside parties or from attackers announcing themselves, and intrusions found internally were caught in roughly 9 to 10 days, versus 25 or more when an outsider made the call. That gap is exactly what NSM exists to close.

A hard assumption sits underneath everything that follows: prevention eventually fails. Firewalls, filters, and endpoint agents stop the bulk of attacks. The ones that get through are exactly the ones worth catching fast, and catching them is the job this guide walks through, from data types to sensors to the encryption problem.

Assumptions Behind NSM

The Assumptions Behind NSM

Doctrine separates NSM from tools that merely raise alerts. In The Tao of Network Security Monitoring, Bejtlich grounded the practice in three assumptions about intruders:

• Some intruders are smarter than you. Skilled attackers can outthink any single control or the people who configured it, so defenders can’t rely on having anticipated every technique in advance.
• Many intruders are unpredictable. New exploits, tools, and methods appear faster than signatures can be written, so a program built only to recognize known attacks will miss the novel ones.
• Prevention eventually fails. Given enough skill, persistence, or luck, something gets through. NSM therefore watches for the intruder already inside rather than trusting the perimeter to hold indefinitely.

Bejtlich’s later work added a threat-centric emphasis. Patching known weaknesses stays necessary, but NSM concentrates on the adversaries themselves: their tactics, techniques, and procedures (TTPs), turned into detections that hold up even when the specific vulnerability changes.

Together, these ideas give defenders something an alert-only stack doesn’t. When a SOC (security operations center) reconstructs how an intruder entered, moved, and escalated, it can write detections for the behavior instead of a single signature, and sophisticated campaigns lose much of their stealth advantage.

Benefits of Network Security Monitoring (NSM)

Value from the practice concentrates where alert-driven tooling runs thin:

• Shorter dwell time. Continuous collection catches the intruder in week 1 instead of month 6, shrinking the window where quiet access turns into real damage.
• Evidence that survives. When an incident lands, the recorded traffic answers what happened, when, and how far it spread, instead of leaving responders to guess from fragments.
• A foundation for hunting. Hypothesis-driven hunts need data to interrogate, and the NSM record is the richest source a hunter gets.
• Regulatory and forensic backing. Investigations, disclosure duties, and legal processes all lean on the traffic history a mature program keeps.

Data NSM Collects

Everything in NSM reduces to data quality, and the classic taxonomy spans 7 types, each answering a different investigative question:

• Full packet capture. Complete copies of traffic, payload included, the ground truth analysts pull when an investigation demands exact answers. Storage costs keep retention short, so full capture works as the deep-dive layer rather than the default.
• Session and flow data. Compact records of who talked to whom, when, over which ports and protocols, and how many bytes moved. Flow records cost little to keep, which makes them the long-memory layer investigations lean on long after packets are gone.
• Transaction data. Protocol-level summaries of what happened inside a connection, from HTTP requests to DNS queries to TLS certificate exchanges. Engines such as Zeek generate these logs, and they carry the bulk of day-to-day analysis.
• Extracted content. Files, executables, and objects carved out of traffic for inspection, feeding sandboxes and malware analysis when something suspicious crosses the wire.
• Alert data. Judgments from detection engines, signature hits from systems such as Suricata, and matches against intelligence feeds- the prompts that tell an analyst where to start looking.
• Statistical and behavioral data. Baselines and deviations built from everything above: a host suddenly beaconing on a schedule, or transfer volumes that dwarf the norm for that machine.
• Metadata. Context about the traffic rather than the traffic itself, such as who owns an IP address or when a domain was registered, which helps analysts judge whether an unfamiliar destination is suspicious.

Fidelity against volume is the trade running through every type. Full capture answers everything and stores nothing for long, flow data stores everything and answers only outlines, and working programs blend the types so each covers another’s blind spot.

Privacy rides along with every byte collected. Full capture records employee and customer content alongside attacker traffic, so disciplined programs scope what they keep, restrict who touches it, and write the policy before the first packet, a tension the field has acknowledged since its earliest tools.

How the NSM Cycle Runs

In motion, the practice follows the cycle its definition names, collection through escalation, with 4 stages repeating continuously:

1. Collection. Sensors placed across the network copy traffic and generate the data types above, passively and continuously, whether anything bad is happening or not.
2. Detection. Signatures catch the known, behavioral analytics catch the abnormal, and intelligence matching catches traffic touching infrastructure already tied to attackers.
3. Analysis. Humans take over where automation stops, using the collected data to reconstruct events, test hypotheses, and separate real intrusions from noise.
4. Escalation and response. Confirmed findings move to containment and cleanup, and everything learned feeds back into new detections.

Threat hunting runs the same cycle in proactive gear. Instead of waiting for a detection to fire, hunters form a hypothesis about what an intruder is doing, then interrogate the data for evidence, and every hunt that finds something becomes a detection that fires automatically next time.

Sensor Placement and Deployment

Where sensors sit decides what a program sees. Perimeter placement watches north-south traffic, everything entering and leaving, while internal placement at data centers and segment boundaries watches east-west traffic, the lateral movement attackers depend on once inside.

Collection happens through taps or SPAN ports. A network tap copies traffic in hardware with full fidelity, while a switch’s SPAN port mirrors traffic more cheaply with some loss under load. Either way, the sensor stays out of band, and a sensor failure never breaks the network.

Clouds complicate the picture. Packet appliances have no rack to live in inside a public cloud, so programs substitute provider mirroring services and flow logs, accepting partial fidelity, and treat the gap as a known limit rather than a surprise.

Placement also depends on knowing what exists. Internet-facing systems IT has lost track of, such as forgotten servers, test environments, and edge devices, never get a sensor, and attackers favor exactly those. External attack surface monitoring such as CloudSEK BeVigil discovers those assets from the outside so they can be brought inside the monitoring plan.

How NSM Differs From IDS, SIEM, NDR, and EDR

• NSM vs. IDS. An intrusion detection system produces alerts from signature matches, one component within the wider practice. NSM wraps that alerting in collection, human analysis, and escalation, then investigates what the IDS flags.
• NSM vs. SIEM. Security information and event management platforms aggregate and correlate logs from everywhere, endpoints to clouds. Network data from NSM sensors feeds the SIEM as one of its richest sources, so the 2 layer rather than compete.
• NSM vs. NDR. Network detection and response names the product category that grew out of the practice, packaging sensors, analytics, and machine learning into platforms. NSM stays the discipline; NDR is one way to buy it.
• NSM vs. EDR. Endpoint detection and response instruments the machines themselves, seeing processes and files no packet reveals. Network sensors return the favor by covering unmanaged devices and anything an agent never reached, which is why mature programs run both.

Encryption Problem

Around 9 in 10 web connections now travel encrypted, per the transparency measurements Google publishes, and that single shift retired the oldest NSM habit: reading payloads. Deep packet inspection tells an analyst nothing when the packet is ciphertext.

What survived is everything around the payload. Connection patterns, timing, and volumes stay visible, TLS handshakes still expose certificates and negotiation details, and fingerprinting techniques in the JA3 and JA4 family identify client software from how it negotiates encryption, no decryption required.

DNS went dark next. DNS-over-HTTPS moved name lookups, once the clearest early-warning channel NSM had, inside encrypted sessions, pushing programs toward resolver logs and toward transaction data as the layer that outlived payload inspection.

Feeding NSM with Threat Intelligence

Sensors see everything and recognize only what they have been told to look for. A connection to a fresh command-and-control server looks like any other TLS session until something labels that destination hostile, and the labels come from outside the network.

The recognition layer comes from intelligence: feeds of command-and-control infrastructure, phishing and malware domains, and actor TTPs convert into detections and watchlists, so the sensor that saw everything starts flagging the traffic that matters. CloudSEK’s Threat Intelligence works on that external side, tracking threat actors, campaigns, and malicious infrastructure across surface, deep, and dark web sources, and handing NSM programs the context their packets lack.

External monitoring also catches what sensors miss entirely. When stolen data or employee credentials surface on dark-web markets or leak sites, CloudSEK XVigil can flag the exposure, often the first sign of an intrusion that internal monitoring never saw.

Neither layer does the other’s job: collection, sensors, and in-network detection belong to NSM tooling, while intelligence explains what the traffic touched, and cyber threat trends plus the attack path an intruder walks decide which detections deserve writing first.

NSM Tools Worth Knowing

Open source built this field, and for many programs the working stack still starts there:

• Analysis engines. Zeek turns raw traffic into the transaction logs that carry daily analysis, a lineage running back to 1995 under its original name, Bro. Suricata supplies signature-based alerting at speed, and the 2 run side by side in countless deployments.
• Full-capture tooling. Arkime indexes and stores complete packet data, giving investigations a replay button for the windows storage allows.
• Flow tooling. Collectors and analyzers for NetFlow and IPFIX records keep the long-memory layer cheap enough to retain.
• Commercial NDR platforms. Vendors package the same data types with managed detection content, machine learning, and support, the buy-side answer for teams without the engineering depth to run the open stack.

Matching tool to team matters more than any single feature. A skilled analyst with open-source logs outperforms an unwatched commercial console, and the reverse holds for teams short on packet fluency.

How to Build an NSM Program

To build an NSM program from scratch, teams must follow 5 key steps:

1. Scope the critical segments. Start where compromise hurts, from payment paths to crown-jewel data stores, since watching everything on day 1 drowns the effort.
2. Pick the collection points. Perimeter and internal chokepoints get sensors, second in order, with taps preferred where fidelity matters.
3. Choose the stack. Open-source engines suit teams with packet fluency, commercial NDR suits teams buying speed, and the third step is an honest reading of which one the team is.
4. Baseline before alerting. Weeks of quiet observation establish normal, so the first tuned detections fire on deviation rather than on noise.
5. Wire in intelligence and hunt on a cadence. Feeds convert into watchlists, hunts run on schedule, and findings loop back as detections.

Frequently Asked Questions

How is NSM different from network performance monitoring?

NSM watches traffic for security threats, while network performance monitoring watches the same links for reliability, bandwidth, and uptime. Tooling overlaps at the collection layer, and the questions asked of the data differ entirely.

Is Wireshark an NSM tool?

Wireshark works as a packet analysis tool inside NSM investigations rather than as a monitoring platform. Analysts open it to dissect captures one conversation at a time, while continuous collection and detection run on dedicated sensors.

What is Security Onion?

Security Onion is a free, open-source Linux distribution that bundles Zeek, Suricata, and analysis dashboards into a ready-made NSM platform. Teams deploy it for monitoring, threat hunting, and log management without assembling the stack piece by piece.

How long is NSM data retained?

NSM data retention scales with cost per type: full packet captures commonly keep 1 to 7 days, transaction logs 3 to 12 months, and flow records a year or longer. Compliance duties and storage budgets set the exact windows.

Can NSM detect insider threats?

Yes, NSM can detect insider threats. East-west visibility surfaces the unusual access patterns, data staging, and large transfers that insiders produce, though pairing network evidence with endpoint and identity signals sharpens attribution.

Where did NSM originate?

NSM originated with Todd Heberlein’s Network Security Monitor, developed at UC Davis and described in 1990 as the first intrusion detection system to analyze network traffic directly. US Air Force computer emergency response teams matured the practice through the 1990s before Richard Bejtlich’s books codified it.