Logging and telemetry for defensive analysis: what the data can answer
Each source of network evidence answers one question well, and leaves the rest open.

Having logs is not the same as being able to answer a question. Each of the four kinds of network evidence answers one question well and leaves the rest to another source, so the craft is to join those sources on time and to refuse to read more out of a record than it can carry. The mechanisms belong to four sibling sheets in this batch: syslog and network logging, SNMP and network metrics, flow telemetry, and packet capture.
Four sources, four questions
The event log records that something happened, in a line carrying a timestamp, a source and a severity: syslog is the protocol “which is used to convey event notification messages” (RFC 5424 lines 65–66). The polled metric is a number read at an interval and describes a state, not an event. The flow record is the metadata of a conversation, and the IPFIX information model “defines the data types and management policy” for those fields (RFC 7012 lines 18–20). The capture is a copy of the wire at one point. The mechanism of each belongs to a sibling sheet; the question it answers is this one’s.
question asked source that answers it what stays open
what did the device report? ──► event log the wire itself
how much, and moving which way? ──► polled metric any single event
who talked to whom, how much? ──► flow record what was said
what crossed this point? ──► packet capture the rest of the path
Time is the join key, and the weakest one
Correlation platforms link related events “by matching IP addresses, timestamps, usernames, and other characteristics” (NIST SP 800-94 lines 3893–3895), and that join is only as trustworthy as the clocks behind it: each host stamps its own entries, and the guide’s example is that “timestamps might indicate that event A happened 45 seconds before event B, when event A actually happened two minutes after event B” (NIST SP 800-92 lines 820–821) — a wrong order, not a constant offset. Where sources record different fields, the join can fail for want of any shared value at all (lines 795–796). Synchronise the clocks, then doubt the time axis first.
Fidelity: a record is a copy, and copies lose things
Every record is a derivation, and each derivation drops something. A metric is not continuous: “Data collection, no matter how frequent, is performed at discrete intervals” (NIST SP 800-137 line 328). A correlation layer adds no truth to what it reads — “SIEM products usually do not generate original event data; instead, they generate meta-events based on their analysis of the imported event data” (NIST SP 800-94 lines 3898–3899) — so its output is an interpretation, not a witness. It keeps only some fields, may introduce errors while normalising the rest, and sits behind a delay (lines 3916–3917, 3924–3925); its one saving property is that the originals stay untouched and can be re-read to check it (lines 3925–3926).
The source is not a witness of itself
The device under observation cannot be the only record of what it did. During an incident “the system’s logs might be altered or destroyed by attackers” (NIST SP 800-92 line 2174), which is why a second copy lives elsewhere; comparing the two shows “what data was changed or removed, which may indicate what the attacker wanted to conceal” (lines 2177–2178). A central store answers the case where the source is compromised exactly when you need it.
Reading with discipline: context, baseline and priorities
“The meaning of an entry often depends upon the context surrounding it” (NIST SP 800-92 line 2328), and context is what is needed “to validate unreliable log entries, such as security software that often generates false positives when looking for malicious activity” (lines 2331–2333). Meaning accrues against a baseline — the goal is to “gain an understanding of the baseline of typical log entries” (lines 2351–2352) — and only a baseline makes a deviation readable. A dashboard is not proof of coverage: it “can provide a false sense of security if used without continued assurance of the relevance of the metrics” (NIST SP 800-137 lines 598–600), and reading controls that were never assessed is “equivalent to using a broken or uncalibrated scale” (line 1467). A product’s severities are no substitute for the organisation’s own, since they “often use inconsistent scales or ratings” (NIST SP 800-92 line 2380). Absence is not evidence: the record that is missing may have been filtered, sampled away, dropped, or never written, so a source answers for what it could have recorded, not for what it happens to hold.
What the network record cannot establish
Network evidence carries no content, no identity and no intent on its own. A flow record shows that two addresses exchanged traffic, not what was sent or who was behind the address; a metric shows a counter moved, not why; a capture shows the packets kept at one point, not the whole path. Where the question concerns the host — a command run, a file written — the answer is host telemetry, another area’s material; the network record only frames when, and between whom.
Retention, integrity, access and cost
Evidence has a lifetime and must be kept as evidence. Retention fixes how far back a question can be asked and must be a decision: log data is often kept for a period that “is relatively short (days or weeks)” (NIST SP 800-92 line 2528). Integrity matters because a log used as evidence must be shown unchanged, so a “message digest” is calculated for each file and stored where it cannot be altered (line 1164). The honest position is not an unbounded archive — the guidance is to require “logging and analyzing the data that is of greatest importance” (lines 194–196) while conceding that some organisations keep “all or nearly all log data generated and stored for at least a short period of time in case it is needed” (lines 196–197). “Collect everything because we can” is that concession misread: an unexamined cost traded for an unexamined promise.
Level and prerequisites
L4 — advanced architectures and network defence. It assumes the four sibling L3 sheets that fix the mechanisms, and the L1 vocabulary of addresses, ports and time. The subject here is the analysis they feed.
Where to go next
References
- RFC 5424 — The Syslog Protocol (Mar 2009, Standards Track); the protocol that conveys event notification messages, Abstract.
- RFC 7011 — Specification of the IP Flow Information Export (IPFIX) Protocol (Sep 2013, Standards Track, STD 77); the export of traffic-flow information from an exporting process to a collecting process, Abstract. Obsoletes RFC 5101, not consulted here.
- RFC 7012 — Information Model for IP Flow Information Export (IPFIX) (Sep 2013, Standards Track); the data types and management policy for the information model, Abstract. Obsoletes RFC 5102, not consulted here.
- NIST SP 800-92 — Guide to Computer Security Log Management (September 2006); the process of generating, transmitting, storing, analyzing and disposing of log data; the challenges of linking sources and of clock accuracy, §2.3.1; log archival, §3.2; storage, retention and integrity, §5.1.2–5.1.3; understanding and prioritising log data, §5.2.
- NIST SP 800-137 — Information Security Continuous Monitoring (ISCM) for Federal Information Systems and Organizations (September 2011); the definition of continuous monitoring and the discreteness of data collection, Chapter One and its footnote 2; metrics, their relevance and the caveat on dashboards, §2.1.
- NIST SP 800-94 — Guide to Intrusion Detection and Prevention Systems (IDPS) (February 2007); the use of SIEM software to correlate logs from several technologies, its normalisation, its delay and its limits, §8.2.2.