DNS security: validating the answer and controlling the resolver
Who may answer a name, and how the answer is proven genuine before anything acts on it.

Every connection starts with a lookup, and the answer arrives unauthenticated by default: a resolver that cannot tell a genuine answer from a forged one hands the client the address an attacker chose. DNS security has two halves: one authenticates the data — DNSSEC proves an answer came from the zone that owns the name, and proves a name’s absence as well as its presence; the other controls the resolver, which server may answer and what it may say.
Why a wrong answer is worse than none
A wrong lookup is worse than a failed one, because nothing begins until it finishes: NIST records that “the first action performed is to use DNS to resolve the domain name of the target system to an IP address” (lines 456–457). Control that answer and the victim reaches the attacker’s server under the correct name; no downstream encryption repairs a session with the wrong host. CISA calls DNS infrastructure “a common threat vector for attack campaigns” (lines 460–461).
What the exchange leaves unprotected
Traditional DNS has neither authentication nor confidentiality. A resolver accepts the first matching answer, so an attacker on the path can reply first and one off the path can guess the transaction details and race the real server. RFC 7873 names that second adversary: an “off-path attacker” “is defined as an attacker who cannot observe the DNS request and response messages between that client and server” (lines 246–248); DNS Cookies are its transport-level mitigation. Beside forged answers sit malicious names resolved faithfully, data smuggled out in query names, and the stub-to-resolver path’s privacy.
stub resolver (client)
│ query
▼
recursive resolver ──▶ validate the answer (DNSSEC) ── chain to a trust anchor? ── no ──▶ SERVFAIL
│ yes
│ apply policy (which names may resolve) ── blocked ──▶ NXDOMAIN / blocked answer
▼
cached answer ──▶ client
Validating the answer: what DNSSEC does
DNSSEC adds origin authentication and integrity to DNS data, and nothing else. RFC 4033 states capability and limit together: the extensions “provide origin authentication and integrity protection for DNS data” (lines 119–120) yet “do not provide confidentiality” (line 121). Each answer carries a signature over the record set; RFC 4034 defines the signature (RRSIG) and authenticated-denial (NSEC) records (lines 40–42). A validator chains a signed answer to a key it already trusts — NIST: “the internet root key should be configured as a trust anchor” (line 1752) — authenticating “both DNS resource records and authoritative DNS error indications” (RFC 4035, lines 43–44), presence and absence alike; NSEC3 resists zone enumeration (RFC 5155). Validation runs on the resolver, not on most clients (line 1848), and oversized replies are retried over TCP (RFC 7766).
Controlling the resolver: which server may answer
The second half is not cryptographic. It is the decision about which resolver a client may use; a client free to reach any resolver bypasses every rule set. NIST’s guidance is to “Block outbound DNS from the internal network to the internet, except for name servers that are authorized to communicate directly with name servers on the internet” (lines 1666–1667) — egress logic applied to port 53. Stub resolvers hold one or two resolver addresses; that is the setting redirect malware changes, and blocking unauthorised outbound DNS denies it any effect.
Response policy: filtering by name
The same resolver can also refuse to answer. NIST describes “DNS firewalls” that “can be implemented in various ways from static or dynamic blocklists to response policy zones (RPZs) and can be seamlessly integrated into the DNS resolution chain” (lines 536–538), enforcing policy “regardless of the features at the client” (lines 548–550). A blocked name is usually answered as NXDOMAIN — “some other form of DNS response to indicate that the domain name does not exist” (lines 558–559). Name-based blocking is thus industry practice, not an RFC-specified protocol in this material.
Encrypted transports and what they cost the monitor
The classic stub-to-resolver exchange is plaintext. Encrypted DNS closes that: RFC 8484 defines DoH as “a protocol for sending DNS queries and getting DNS responses over HTTPS” (rfc8484.txt lines 18–19), and RFC 7858’s premise is that “Encryption provided by TLS eliminates opportunities for eavesdropping and on-path tampering with DNS queries in the network” (lines 25–27). But the encryption that protects the query also removes it from a passive observer — troubleshooting staff “will not have ready access to the contents of DNS queries or responses” (lines 1646–1647). A network watching DNS must make the resolver the observation point, not the wire.
Logging: the first point to instrument
Because every conversation passes through a lookup, the resolver’s log is the earliest evidence a network produces. NIST sets a floor: “DNS queries and the responses associated with domains that are classified as malicious or unauthorized by protective DNS services should always be logged” (lines 621–622). Those hits are kept and forwarded to the monitoring platform.
Limits and the common mistake
- DNSSEC is not a reputation system. A valid signature proves the name resolved as its owner published it, not that the owner is trustworthy; and it never hides the query. It was “designed to protect the data itself from tampering” (line 1248) and “does not protect the confidentiality of DNS query/response data” (lines 1258–1259).
- Signed is not the same as validated. If every algorithm a zone signs with is unknown to your validator, RFC 6840 says “the zone is treated as if it were unsigned” (
rfc6840.txtlines 442–444). - A blocked name looks like a missing name. At the client a policy NXDOMAIN is indistinguishable from nonexistence, and a DNSSEC failure arrives as a generic error.
Level and prerequisites
L4 — architecture and defence. It assumes the L1 sheet on names, zones and resolution and the L2 sheet on records: here resolution is not re-taught, it is trusted or distrusted. It builds the two halves of DNS security conceptually — no product is configured, no output reproduced.
Where to go next
References
- NIST SP 800-81r3 — Secure Domain Name System (DNS) Deployment Guide (Rose, Liu, Gibson; March 2026): DNS as a control point, protective DNS, encrypted DNS, DNSSEC validation and trust anchors, stub-resolver limits, logging.
- RFC 4033 — DNS Security Introduction and Requirements (Standards Track, March 2005): origin authentication, integrity, and the no-confidentiality boundary.
- RFC 4034 — Resource Records for the DNS Security Extensions (Standards Track, March 2005): RRSIG, DNSKEY, and authenticated denial of existence (NSEC).
- RFC 4035 — Protocol Modifications for the DNS Security Extensions (Standards Track, March 2005): the security-aware resolver authenticating records and error indications.
- RFC 5155 — DNS Security (DNSSEC) Hashed Authenticated Denial of Existence (Standards Track, March 2008): NSEC3 and zone enumeration.
- RFC 6840 — Clarifications and Implementation Notes for DNS Security (DNSSEC) (Standards Track, February 2013): unsupported algorithms and the unsigned-zone case.
- RFC 7766 — DNS Transport over TCP — Implementation Requirements (Standards Track, March 2016): truncation, the TC flag and retry over TCP.
- RFC 7858 — Specification for DNS over Transport Layer Security (TLS) (Standards Track, May 2016): privacy on the stub-to-resolver path.
- RFC 7873 — Domain Name System (DNS) Cookies (Standards Track, May 2016): off-path attackers and cache poisoning.
- RFC 8484 — DNS Queries over HTTPS (DoH) (Standards Track, October 2018): DNS mapped into an HTTPS exchange.