DNS Security: DNS Filtering, DNSSEC and DNS over HTTPS

Why DNS is a security concern

The Domain Name System (DNS) translates human-readable domain names (www.example.com) into IP addresses (203.0.113.42) that computers use to communicate. Every website, every email, every cloud application, every API call begins with a DNS query. DNS is so fundamental that it is often taken for granted and overlooked in security assessments.

This is exactly what makes DNS a preferred attack target. Attackers exploit DNS in three ways:

DNS as an attack vector. Manipulation of DNS responses to redirect users to malicious websites (DNS spoofing, DNS hijacking). The user types the correct URL but lands on a phishing page.

DNS as a data channel. DNS tunnelling uses DNS queries and responses to exfiltrate data from the network or send commands to malware. Since DNS traffic is unrestricted in most networks, it goes unnoticed.

DNS as an intelligence source. Attackers use DNS queries to gather information about a company's infrastructure: subdomains, mail servers, IP ranges, services in use.

DNS threats in detail

DNS spoofing and cache poisoning

In DNS spoofing, an attacker forges DNS responses so that a domain resolves to a different IP address than the correct one. DNS cache poisoning poisons the cache of a DNS resolver with forged entries. Every user who subsequently uses that resolver receives the manipulated response.

An example: the attacker poisons your resolver's DNS cache so that your bank's domain points to the IP address of a perfectly cloned phishing page. The user sees the correct URL in the address bar, the phishing page's SSL certificate looks correct at first glance (the attackers have obtained a Let's Encrypt certificate for a similar domain), and the user enters their credentials.

DNS hijacking

In DNS hijacking, the attacker takes control of DNS records, either by compromising the DNS server, by accessing the registrar account (where the domain is registered) or by manipulation in transit.

Unlike cache poisoning, the authoritative DNS source is manipulated here, not just a cache. The impact is broader and harder to detect.

DNS tunnelling

DNS tunnelling encodes data in DNS queries and responses. A compromised system in the network sends DNS queries to an attacker-controlled domain (e.g. data.evil-domain.com), with the data to be exfiltrated encoded in the subdomain portion (e.g. base64-encoded-data.evil-domain.com). The attacker's DNS server decodes the data and can send commands back via the DNS response.

DNS tunnelling is particularly dangerous because DNS traffic is not filtered in most enterprise networks, and firewalls let DNS queries through by default.

DDoS attacks via DNS

DNS amplification attacks abuse open DNS resolvers to amplify DDoS attacks. The attacker sends DNS queries with a forged source address (the victim's address) to open resolvers. The resolvers send their responses (which are significantly larger than the queries) to the victim. This allows an attacker to direct massive data volumes at a target with little bandwidth of their own.

DNS filtering: first line of defence

DNS filtering is the simplest yet one of the most effective measures for improving network security. The principle: every DNS query is checked against a database of known malicious domains. If the queried domain is known to be malicious (malware, phishing, command and control), the query is blocked.

Why DNS filtering is so effective

The vast majority of malware and phishing attacks use domain names for communication. Ransomware must contact its command-and-control server. A phishing email contains a link to a malicious domain. A drive-by download is delivered from a compromised domain. If DNS resolution of these domains is blocked, the attacks are prevented at their origin.

Studies show that DNS filtering can prevent up to 33% of all malware infections -- without an agent on the client, without signature updates on the endpoints and without user interaction.

DNS filtering options

Option 1: Public DNS resolver with filtering. The simplest option. You configure your network's DNS servers to a resolver with integrated filtering:

• Quad9 (9.9.9.9): Non-profit, blocks known malware domains, no logging, DSGVO (GDPR)-compliant (headquartered in Switzerland). Recommended for privacy-conscious companies.
• Cloudflare for Families (1.1.1.2 / 1.1.1.3): Malware blocking (1.1.1.2) or malware + adult content (1.1.1.3). Fast, but a US company.
• CleanBrowsing: Various filter levels, including an option specifically for enterprises.

Option 2: Managed DNS filtering service. More control and reporting. You use a specialised service that offers a dashboard with statistics, custom block lists and per-user-group policies:

• Cisco Umbrella (formerly OpenDNS): Enterprise solution with extensive reporting, integration into the Cisco security ecosystem.
• Cloudflare Gateway: Part of the Cloudflare Zero Trust platform, granular policies, logging.
• DNSFilter: Focused on SMEs, easy setup, good value for money.
• NextDNS: Flexible, privacy-friendly, extensive configuration options.

Option 3: Local DNS resolver with filtering. You run your own DNS resolver with filtering capability. Technically more demanding but with full control over the data:

• Pi-hole: Open source, runs on a Raspberry Pi or in a Docker container. Primarily known for ad blocking but also usable for malware filtering.
• AdGuard Home: Similar to Pi-hole but with an HTTPS interface and DoH/DoT support.
• Unbound + Response Policy Zones (RPZ): Technically demanding but maximally flexible.

Implementing DNS filtering

The simplest implementation for an SME:

1. Configure your internal DNS server (typically the domain controller) to use a DNS filtering service as forwarder (e.g. Quad9)
2. Block all DNS traffic (port 53 UDP/TCP) from internal clients directly to the internet on the firewall. Only the internal DNS server may send DNS queries externally.
3. Block DNS over HTTPS (port 443) to known DoH endpoints so clients cannot bypass DNS filtering (more on this in the DoH section)
4. Monitor DNS logs for unusual patterns (high query rates, long subdomain names, queries to unknown top-level domains). In ISMS Lite, you document DNS filtering as a technical control in your risk management and track regular reviews.

DNSSEC: integrity protection for DNS

DNSSEC (Domain Name System Security Extensions) extends DNS with cryptographic signatures that ensure the integrity and authenticity of DNS responses. DNSSEC prevents DNS spoofing and cache poisoning because the resolver can verify from the signature whether a DNS response actually originates from the authoritative DNS server and has not been tampered with.

How DNSSEC works

Each DNS zone (e.g. example.com) is signed with a cryptographic key pair. The Zone Signing Key (ZSK) signs the individual DNS records. The Key Signing Key (KSK) signs the ZSK. A chain of trust links the signatures from the root zone through the TLD (.com) to your own domain.

When a DNS resolver receives a DNSSEC-signed response, it verifies the signature along the chain of trust. If the signature does not match (because the response was tampered with), the resolver discards the response and reports an error.

Using DNSSEC as a resolver

As a company, you do not have to implement DNSSEC for your own domains (although it is recommended), but you should ensure your DNS resolver performs DNSSEC validation. This means your resolver checks the DNSSEC signatures of incoming responses and discards tampered responses.

Most public DNS resolvers (Quad9, Cloudflare, Google Public DNS) perform DNSSEC validation by default. If you run your own resolver (Unbound, BIND), you must explicitly enable DNSSEC validation.

Enabling DNSSEC for your own domains

If you operate your own domains (e.g. for your website or email), you should enable DNSSEC. The process:

1. Generate a key pair (KSK and ZSK) on your DNS server or with your DNS hosting provider
2. Sign the DNS zone with the ZSK
3. Deposit the DS record (a hash of the KSK) with your domain registrar
4. Test the DNSSEC configuration with tools like dnsviz.net or verisignlabs.com/dnssec/

Many DNS hosting providers (Cloudflare, Hetzner, AWS Route 53) offer DNSSEC as a one-click activation.

Limitations of DNSSEC

DNSSEC protects integrity but not confidentiality. DNS queries and responses are still transmitted in plain text and can be intercepted by anyone who can read the network traffic (internet service provider, network administrator, attacker on the local network). For encrypting DNS traffic, you need DoH or DoT.

DNS over HTTPS and DNS over TLS: encrypted DNS traffic

DNS over TLS (DoT)

DNS over TLS (RFC 7858) encrypts DNS queries and responses over a TLS connection on port 853. The client's resolver establishes an encrypted connection to the upstream DNS server and sends all DNS queries over this connection.

Advantage: Easy to identify and control (dedicated port 853), good performance. Disadvantage: Easily blockable (block port 853), the ISP or network administrator can detect that DoT is being used (even though they cannot read the content).

DNS over HTTPS (DoH)

DNS over HTTPS (RFC 8484) encrypts DNS queries and responses over HTTPS on port 443, the same port as regular web traffic. DNS traffic is indistinguishable from normal HTTPS traffic.

Advantage: Cannot be blocked without also blocking normal HTTPS traffic. Protection against censorship and surveillance. Disadvantage: Problematic for enterprises because DoH undermines central DNS control (more on this shortly).

DoH in the enterprise: curse or blessing?

This is where it gets complicated for enterprises. Browsers like Firefox and Chrome partially activate DoH automatically and send DNS queries directly to Cloudflare or Google, bypassing the internal DNS server and thus also bypassing DNS filtering.

This means: an employee can access malware domains despite DNS filtering in the corporate network if their browser sends DoH to Cloudflare and the DNS query is not captured by corporate DNS filtering.

Recommended strategy for enterprises:

1. Disable DoH in the browser: Via GPO (Group Policy), disable automatic DoH activation in Firefox and Chrome. Firefox automatically detects "managed networks" and disables DoH when an enterprise policy is present.

2. Use DoH/DoT between internal DNS and upstream: Your internal DNS server (which performs the filtering) communicates via DoT or DoH with its upstream resolver. This way, DNS traffic between your network and the internet is encrypted, but internal filtering remains effective.

3. Block known DoH endpoints on the firewall: Block access to the IP addresses of known DoH providers (Cloudflare, Google, Quad9) on port 443 for all clients except the internal DNS server.

4. Block DoT on the firewall: Block port 853 for all clients except the internal DNS server.

Detecting and preventing DNS tunnelling

DNS tunnelling is one of the most sophisticated exfiltration techniques and difficult to detect because DNS traffic is generally considered harmless. But there are indicators:

Unusually long subdomain names. Normal DNS queries have short subdomains (www, mail, ftp). DNS tunnelling uses long, random-looking subdomains (aGVsbG8gd29ybGQ.evil-domain.com) to transport as much data as possible per query.

High query rate to individual domains. If a client sends thousands of DNS queries to the same domain within an hour, this is a strong indicator of DNS tunnelling.

Unusual record types. DNS tunnelling often uses TXT records or CNAME records, which can transport more data than A records.

Queries to unknown or newly registered domains. Most DNS tunnels use specially registered domains that do not appear in any threat intelligence database.

Countermeasures

• Limit DNS queries to a maximum subdomain length
• Rate-limit DNS queries per client and domain
• Analyse DNS traffic with specialised tools (dnstap, Zeek/Bro, Suricata)
• Allow DNS queries only to defined resolvers (no direct DNS traffic to the internet)
• Deploy next-generation firewalls with DNS tunnelling detection (complementing IDS/IPS)

DNS architecture for SMEs: recommendation

A sensible DNS architecture for an SME with 50 to 500 employees:

1. Internal DNS server (Active Directory DNS or Unbound): resolution of internal domains, caching, DNSSEC validation
2. DNS forwarding to a DNS filtering service (Quad9, Cloudflare Gateway, Cisco Umbrella): malware and phishing blocking
3. DoT/DoH between internal DNS server and filtering service: encryption of external DNS traffic
4. Firewall rules: Only the internal DNS server may send DNS traffic externally. All other DNS queries (port 53, port 853) are blocked.
5. DoH disabled in browsers via GPO
6. DNS logging on the internal DNS server, forwarding to central log management

Further reading

• Firewall configuration for SMEs: rules, zones and best practices
• Network segmentation for SMEs: why and how to segment your network
• Wi-Fi security in the enterprise: WPA3, 802.1X and guest networks
• Intrusion detection and prevention (IDS/IPS) for mid-market companies
• Email security: configuring SPF, DKIM and DMARC correctly

DNS security is not a luxury topic for large enterprises. DNS filtering as a first line of defence, DNSSEC validation as integrity protection and a controlled DNS architecture with a central resolver are measures that any SME can implement with manageable effort and that deliver measurable security improvements.