What broke

A handful of static sites run on one public web host, each on its own subdomain. Adding a site means creating a DNS record for the new name at the DNS provider, then giving the web server a block for it. This case is the DNS half.

Before creating the record, the obvious check ran: does the name already exist? A lookup answered NXDOMAIN — the DNS reply that means "no such name" — so the name was free. The A record was created. Public resolvers answered with the new address within a minute. Every server-side check passed: curl from the host, the certificate, a public DNS-over-HTTPS query.

Two and a half hours later, the person the site was for still could not load it.

How it showed

Their browser reported DNS name does not exist, for a name every public resolver had been answering for two hours. A raw UDP query to their home router — the resolver every device in that house asks first — returned rcode=NXDOMAIN for the new name and NOERROR for its sibling in the same zone. The same query to 1.1.1.1 returned the address.

The earlier existence check had gone through that same router. Resolvers cache "no such name" answers as well as real ones, for as long as the zone says they may — the negative TTL, 300 seconds here. The router cached the negative answer and held it past those 300 seconds; consumer resolvers do not reliably honor the SOA minimum the way the RFC assumes. Every check that ran after the record was created queried a different resolver, so each one reported success to an operator while this one router kept answering NXDOMAIN to the user.

What was watching

The synthetic monitoring sensor — scripted probes that exercise the system from outside — was on the path. Every check that ran after the record was created answered from a server or a public resolver, neither of which held the earlier negative cache entry. The probe existed; its vantage point was wrong; it did not take the user's path through their home router.

A customer caught the failure two and a half hours after the record was created, when their browser reported the name did not exist. Nothing instrumented caught it before them.

Where it sits

This is a change — a new DNS record — verified from the wrong vantage point, producing a runtime failure that existed only on the path the real user took. The synthetic monitoring sensor names this directly: a probe that exercises a system the way a user would, from outside it. The checks that ran here were synthetic in form and wrong in vantage — each was a real request from a resolver with no record of the negative answer cached on the user's own path.

This is the second of two cases in this casebook with the same shape: a pre-flight check that is not read-only. See Validating a web server's config only reads it, where a config validator wrote a log file the way this existence check wrote a cache entry — both created state the real run then tripped over, and both reported success from inside the check that had just caused the problem.

What we did

The router's cache had to be flushed by the person affected, by hand. The order of operations for every new name since has changed: check for absence against the authoritative nameservers only (dig @ns1.<provider>, which answers from the zone rather than from a cache), create the record, verify against a public resolver, and query the user's own resolver last — only to read what it already believes, never to test whether something exists.

What now fails loudly

• Never probe for absence through a path that remembers. A lookup that

misses creates a cache entry wherever it passed through, and that entry outlives the fix. Existence checks go to an authority, not to a caching resolver.

• Verify from the user's vantage, not the operator's. A check that

succeeds from the server proves the server can reach itself. The check that matters takes the same path the user will.