Websites and safe changes

Change DNS records safely

Prepare a reversible record change, account for the old TTL, verify both endpoints, and keep the previous service available during the transition.

A DNS change is easy to make and slow to take back. Once an answer is cached, you can’t recall it. So do most of the work before you touch anything.

Lower the TTL first, and early

Write down the current records and their TTLs. Then lower the TTL on the record you’re about to change.

Lowering it doesn’t touch answers already cached with the old value. If the TTL is 24 hours, resolvers may hold the old answer for up to 24 hours after you lower it. So lower it at least one full old-TTL before the move. Note the original value so you can restore it later.

Test the new service before DNS knows about it

Set up the new server completely while public DNS still points at the old one. Application, certificate, redirects, data.

Then send a real HTTPS request to the new address while keeping the real hostname:

curl --resolve www.example.com:443:203.0.113.40 \
  https://www.example.com/

This tells curl “for www.example.com on port 443, connect to 203.0.113.40”. TLS and the Host header use the real hostname, so you test exactly what visitors will get. Public DNS is untouched.

Write the plan down

I always use a short checklist for this:

  1. Export or note the current A, AAAA, CNAME, MX, and important TXT records.
  2. Write the expected new answer and the condition that triggers a rollback.
  3. Lower the TTL early and wait out the old TTL.
  4. Test the new service with curl --resolve.
  5. Change the authoritative record.
  6. Query an authoritative server directly, then compare a few recursive resolvers.
  7. Keep the old service running until old positive and negative caches have expired.
  8. Restore a normal TTL once everything is stable.

Step 7 is the one people skip. Some visitors keep landing on the old server for hours. It has to keep working for them.

Nameserver changes are bigger

Changing nameservers moves the whole zone, and the parent delegation has its own caches.

If the domain uses DNSSEC, be extra careful. The DS record in the parent must match the keys in the active zone. A stale DS pointing at a zone that’s now unsigned, or signed with different keys, makes validating resolvers return SERVFAIL for everything, while non-validating tests look healthy.

Don’t trust your laptop

Your machine has its own caches: operating system, browser, maybe the router. “It works for me” proves very little. Query named resolvers with dig @ and ask authority directly.

Try this: write a reversible runbook for moving one test hostname. Include the old-TTL deadline, the curl --resolve preflight, the authority query, two resolver queries, the rollback trigger, and when the old endpoint can be shut down.

Lesson completed