On the evening of 5 May 2026, the servers of thousands of German online shops were running perfectly: the shop software responded, the database was healthy, the TLS certificate was valid, the checkout was intact. Yet a substantial share of customers could no longer get through -- their browsers simply reported that the site was unreachable. The cause was not in any single shop but one layer below: a zone signing key rollover in the .de zone had failed. From 21:43 CEST (heise online) onwards, the registry was signing with a key whose public counterpart had not been published correctly, and for roughly three hours (DENIC eG) DNSSEC-validating resolvers returned an error instead of an address for .de domains. Anyone running uptime monitoring from their own location may well have seen nothing but green. That is precisely the lesson of the incident: name resolution is a maintenance object in its own right, with its own failure modes -- and it belongs in monitoring that checks more than a status code. This article covers the signature chain, domain locks, expiry dates, TTL planning, resolver-aware observation and a runbook for disruptions that originate outside your own remit.
The night everything worked and nothing arrived
The sequence is well documented and worth examining closely, because it shows a failure mode that appears in no conventional maintenance plan. On 2 May 2026 (heise online), DENIC began the scheduled rotation of the zone signing key for the .de zone. A new public key with the ID 33834 (heise online) was published -- three days before it was first used for signing. That waiting period is deliberate: resolvers around the world should know the new key before signatures made with it appear. On 5 May at 21:43 CEST (heise online), key 33834 showed up for the first time in a signature for the SOA record of the .de zone. From that moment the signature chain was broken.
The cause was neither an attack nor a hardware failure, but a defect in in-house developed code. Instead of generating a single key pair and loading it into all connected hardware security modules, the software created one key pair per HSM (DENIC eG) -- three different private keys, all carrying the same identifier 33834, but only one of them matching the published public key. Since all signing systems were in use, the zone served signatures from two non-matching keys and one matching key. As a result, only around one third (DENIC eG) of the RRSIG records remained validatable. The signing environment in question had only been put into service in April 2026 (DENIC eG) as the third generation. According to DENIC, the defective section of code was "not fully covered by the test scenarios" (DENIC eG) and therefore went unnoticed in test runs and in cold parallel operation before commissioning.
The outage also hit shops with no DNSSEC at all
Recovery took time. Only at around 01:17 CEST (heise online) was the decision taken to roll back to the previous key 32911; at 00:08 CEST (DENIC eG) DENIC had already started distributing a corrected zone, and at 01:15 CEST (DENIC eG) the state of operations prior to the outage was fully restored. In its final report of 11 June 2026 (DENIC eG), DENIC announced measures including enhanced alerting, accelerated zone backups, validation of the zone before delivery, suspension of further ZSK rollovers pending improved procedures, and an external security and process analysis. For shop operators, though, the decisive insight is a different one: between your server and your customer sits an infrastructure you do not control, whose failure looks exactly like a total outage of your own shop.
Why the uptime dashboard stayed green
The real subtlety of the incident was its uneven distribution. Whether a customer reached the shop depended not on the shop but on which DNS resolver their device used. Resolvers with active DNSSEC validation check every answer against the signature chain. If the check fails, the answer is deemed bogus and discarded -- the client receives SERVFAIL, meaning resolution failed. Resolvers without validation skip that check and pass the answer through. For them, the shop was reachable the whole time. "Non-validating resolvers" (DENIC eG) continued to operate normally.
The customer base therefore split into two groups without any technical signal appearing in the shop. Those using a validating corporate or public resolver saw a total outage. Those using their connection's default resolver, if it did not validate, kept buying. Matters were complicated further because some large resolver operators "temporarily suspended validation of .de domains" (DENIC eG) -- which softened the impact for their users but confused diagnosis for operators even more: the same domain was reachable or not depending on the query path and the time of day.
| Check layer | Question asked | Answer on 5 May 2026 | Blind spot |
|---|---|---|---|
| HTTP uptime check from your own location | Is the server responding? | Green -- HTTP 200 | Says nothing about whether the domain resolves for the customer |
| HTTP check via a non-validating resolver | Is the server responding? | Green -- HTTP 200 | Does not check the signature chain and does not report the break |
| DNS check via a validating resolver | Is the answer cryptographically provable? | Red -- SERVFAIL | Detects the break but says nothing about the shop itself |
| Chain and expiry check | Do DS, DNSKEY and RRSIG match, and for how much longer? | Red -- chain broken | Requires dedicated check logic instead of a status code |
From this follows an uncomfortable but useful rule: a single check from a single location with a single resolver describes reachability for exactly one query path -- not for the market. Anyone wanting to know whether their shop can sell must reproduce the customer's path, and that includes the resolver question just as much as region and device. What a solid HTTP foundation looks like is described in the article on setting up uptime monitoring; the DNS layer discussed here sits beneath it and is not covered by it.
The signature chain as a maintenance object
DNSSEC is not an exotic extra. With RFC 9364 (IETF RFC 9364), published in February 2023 and simultaneously designated BCP 237, the IETF recorded that using DNSSEC for origin authentication of DNS data is current best practice. The German BSI likewise recommends securing DNS information with DNSSEC in its IT-Grundschutz module APP.3.6 DNS Server (BSI). The technology solves a real problem: without a signature, a DNS answer can be forged, and whoever forges the answer diverts the customer to a foreign server while the address bar still shows the correct domain. The price is that a broken signature is stricter than no signature at all -- it leads to the answer being discarded rather than to a warning.
That is exactly why the chain belongs in maintenance. It consists of a handful of building blocks that must fit together after every change of registrar, DNS operator or hosting. The most dangerous moment is not routine operation but the switch: a provider migration in which the DS record in the parent zone points to a key the new operator no longer uses renders the domain unreachable for validating resolvers -- with a technically flawless shop behind it.
DNSKEY
The zone's public key. It has to match the private key actually used for signing. This is precisely where the break occurred on 5 May: one key was published, while signing was partly done with others.
RRSIG
The signature over a record set. It carries an expiry date. If it expires without being renewed, validating resolvers discard the answer -- a silent outage on a timer.
DS record
The fingerprint of the key, held in the parent zone via the registrar. It connects your zone to the chain of trust. After every key change it has to be updated to match.
NSEC3
The cryptographic proof that a record does not exist. If that proof breaks, the absence of a DS record becomes unprovable too -- which dragged unsigned domains into the outage on 5 May.
Key rollovers
ZSK and KSK rollovers are routine but error-prone. New keys must be published before they sign, and old ones stay valid until the last cached signature has expired.
Provider migration
The single most critical operation. DNS operator, registrar and DS record have to be switched in the right order. A DS record pointing to a key no longer in use locks you out of your own domain.
A checkpoint after every change to domain or hosting
Registrar and registry lock: sealing the domain against changes
The second area of DNS maintenance has nothing to do with cryptography and everything to do with access rights. A domain is not property in a physical sense but an entry in a database -- and whoever may change that entry controls the shop. A compromised registrar account is enough to redirect name servers, route mail to foreign servers and transfer the domain away. The shop keeps running throughout; it is simply no longer reachable under its own address.
ICANN's security bodies have described a graduated approach to this for years. Locks are formally status codes that "prevent changes to your domain name registrations, and block attempts to transfer or delete your domain names" (ICANN, SAC 044). Beyond that, several registry operators offer a registry lock that applies "in addition to lock services offered by registrars" (ICANN) and frequently involves manual approval -- a human being who confirms a change outside the automated path. That friction is exactly the point: it costs a few hours during a planned migration and prevents a change nobody ordered.
| Protection layer | Protects against | Can be lifted by | Recommended for |
|---|---|---|---|
| Registrar lock (client status codes) | unintended transfers and deletions | the holder in the registrar portal, usually instantly | every domain in productive use |
| Registry lock | changes made via a compromised registrar account | a defined process with manual approval | shop, payment and mail domains |
| Two-factor on the registrar account | account takeover by password alone | the holder with a second factor | every account with authority over domains |
| Separate accounts per purpose | collateral damage from one compromised login | role and permission assignment | agencies and larger teams |
Prioritisation follows damage, not traffic. A domain used to process payments or send order confirmations deserves the stronger protection -- even if it has fewer visitors than the main domain. Anyone already maintaining their mail records will recognise the pattern from the article on DMARC and shop email deliverability: that one is about who may send in your name. This one is about the layer above -- who determines where your domain points at all. That access is the actual attack surface holds for domains just as it does for the supply chain of a shop installation; the parallel is explored in the article on supply chain attacks via shop dependencies.
Expiry dates: the quietest outages in the calendar
Domains expire. That sounds trivial and is nonetheless a recurring cause of total outages, because the expiry date lives somewhere nobody watches: in a registrar interface, tied to a credit card reissued two years ago, with a notification address pointing at an employee who left. With 18 million (DENIC eG, June 2026) registered .de domains, the administrative load in many companies has grown over years and is rarely documented. ICANN's guidance for registrants therefore stresses maintaining the contact data of the registration account as a protective measure in its own right (ICANN, SAC 044) -- because a warning email nobody reads is not a warning.
- Expiry date of every domain monitored automatically, with at least 60 days of lead time and escalation to a role rather than a person.
- Payment method at the registrar checked for validity -- an expired card turns automatic renewal into automatic deletion.
- Contact addresses of the registration account pointed at a distribution list or role address that outlives individual departures.
- Holder data accurate and current -- in a dispute or a recovery, what counts is who is demonstrably on record.
- Secondary domains, typo variants and old campaign domains inventoried: they often redirect to the shop and fall into other hands when they lapse.
- Expiry of DNSSEC signatures and validity of the DS record in the same monitoring as domain and certificate lifetimes.
An inventory is the prerequisite, not the finishing touch
TTL planning before migrations
The time to live of a DNS record determines how long resolvers may cache an answer. In routine operation a high value is pleasant: fewer queries, faster answers, less load. During a migration that same value becomes the problem, because it also determines how long the world still points at the old server after you have long since switched. Migrating with a TTL of 24 hours means a full day with two live environments -- and orders landing on the old server.
Sequence therefore matters more than speed. An orderly switch lowers the TTL, waits out the old TTL, performs the change, observes and only then raises the value again. The DENIC incident shows the same mechanics from the other direction: because caches still held the old, valid answer for a while, the fault did not hit all users at once but seeped in over minutes -- and receded just as unevenly after the correction.
- At least 48 hours before the planned migration, lower the TTL of the affected records to a low value such as 300 seconds.
- Let the old, high TTL expire completely before the actual switch begins -- otherwise the reduction has not taken effect yet.
- Test the target environment via its IP or a temporary name before the switch, including checkout and payment return path.
- For signed zones, coordinate the DS record and the key change before the name server changes -- not afterwards.
- Switch over and observe both environments in parallel for as long as requests still arrive at the old address.
- Raise the TTL again only once the migration has been signed off and the fallback path is no longer needed.
Lowering the TTL belongs in the window plan
Resolver-aware monitoring instead of a single check
When a single check fails to reflect reality, the answer is not "more checks" but "different questions". The DNS layer needs checks that make the difference between validating and non-validating resolution paths visible -- otherwise the pattern of 5 May repeats itself: the dashboard stays green while some customers cannot find the shop. Three checks cover the core, and none of them is an HTTP status code.
Two resolution paths in parallel
Query the same domain via a validating and a non-validating resolver. If the results diverge, the fault lies in the signature chain -- regardless of whether the shop is responding.
Chain and remaining validity
Check whether DS record, DNSKEY and RRSIG match, and how long the current signatures remain valid. This catches a break after a provider migration before a customer reports it.
Multiple regions
Query resolution from different networks. Regional differences in the resolver landscape mean a fault can be immediately visible in one place and invisible in another.
Expiry dates as a metric
Treat domain expiry, signature validity and certificate lifetime as monitored values with lead-time thresholds -- not as calendar entries that get lost when responsibilities change hands.
Lock and record drift
Compare status codes, name server entries and key records against a defined target state on a regular basis. An unexpected change is either a mistake or an attack -- you want to know about both the same day.
An alert path off the domain
Alerting that does not run through the monitored domain. If name resolution fails, so does mail to mail@your-domain.de -- the notification needs an independent route.
The last point is routinely overlooked and was especially relevant on 5 May: an alerting path that itself depends on the disrupted zone goes silent exactly when it is needed. The same logic applies to the status page you would like to point customers at -- if it sits under the same domain, it is unreachable in an emergency too. How checks extend all the way into the order path is shown in the article on checkout and transaction monitoring; the DNS layer is the stage before it, because without resolution no customer reaches the checkout at all. Together with monitoring of SSL certificates, this produces an unbroken chain from name resolution to the encrypted connection.
A green uptime signal proves that your server answers -- not that your customers can find it.
A runbook for disruptions that are not yours
The most unpleasant part of the DENIC incident was the powerlessness. There was no switch in your own shop that would have fixed the problem, because the fault sat in a zone nobody but the registry can change. Even so, the difference between prepared and unprepared is substantial -- it decides whether you spend two hours dismantling your own infrastructure or know within ten minutes that the fault lies elsewhere, and invest the remaining time in communication.
- Narrow it down: does the server respond via its IP address? If so, the shop is healthy and the problem is in resolution -- searching further inside the shop only costs time.
- Differentiate: query the same domain via a validating and a non-validating resolver. If only the non-validating one answers, the signature chain is the suspect.
- Locate it: does it affect only your domain, or other domains under the same suffix too? If third-party .de domains are affected as well, the cause sits above your zone.
- Check the registry status: status notices from the registry operator are the most reliable source -- DENIC reported the disruption during the night and later followed up with an analysis and a final report.
- Communicate: inform customers and support through a channel that does not sit under the affected domain. An honest status notice is worth more than trying to hide someone else's problem.
- Document: record timestamps, symptoms, query results and decisions -- for later evaluation and as evidence towards third parties.
What does not help during a registry disruption
The rest is a question of organisation: who reaches whom at 10 pm, who decides, who talks to customers? That is the same structure that carries a security incident, and the one that emergency support with defined response times formalises. What such a sequence looks like in practice is also described in the article on the emergency plan for a hacked website -- the cause differs, the approach is similar.
DNS as a fixed item in maintenance
In most maintenance contracts the DNS layer does not appear. It falls between responsibilities: the host owns the server, the agency owns the shop, the registrar owns the domain -- and nobody owns the signature chain until it breaks. That is not negligence but a consequence of history: the layer was inconspicuous for a long time, and what is inconspicuous ends up in no scope of work. 5 May 2026 corrected that assumption.
In practice this means three additions. First, a maintained inventory of all domains with registrar, DNS operator, expiry date, lock status and DNSSEC state. Second, monitoring that checks resolution across different resolvers, reconciles the signature chain against the DS record and treats expiry dates as metrics rather than calendar entries -- the same approach our monitoring applies to reachability, certificates and the order path. Third, a documented procedure for cases where the cause sits above your own zone, including an alert path and status communication outside the affected domain.
What makes this binding is the agreement, not the technology. An SLA maintenance contract records which checks run, how quickly a response follows and who decides in an emergency; the individual building blocks are described in our maintenance services. Which response times are realistic and sensible is put in context by the article on response times in the SLA. The claim here is deliberately sober: a disruption at the registry cannot be prevented from outside. But whether you understand it in minutes or in hours, whether your customers get an explanation, and whether your own chain still holds after the next provider migration -- those are very much in your hands.
Sources and studies