Last Wednesday I was poking around in the records on our primary DNS server – setting up a new domain, adding some addresses to our reverse-lookup zone, and doing some general housekeeping. During the housekeeping part I spotted a couple of aliases in the hartfordschools.org domain records: routerguy and shadowrun. These were a pair of Mac boxes that
majikshop had setup back in '99 to do firewall/DNS work. We took them out of service in '02 and replaced them with a pair of much more mundane UNIX DNS servers (with much more mundane names: ns1.hartfordschools.org and ns2.hartfordschools.org). At the time we replaced them, I flipped their name entries in the DNS records from being "A" records (hosts) to being "CNAME" records – aliases that pointed to the new DNS servers. Wednesday night I nuked those aliases. To understand why this was a bad thing, you have to understand a little bit about how DNS works. DNS is a hierarchy – at the top are the "root" servers. They know about the servers that handle the Top Level Domains (TLDs). The TLDs are .com, .org, .edu, and all of those ISO country codes – .uk, .ru and the like. The DNS servers responsible for each TLD know about the DNS servers for the domains within that TLD. EG, when you key in www.coke.com into your web browser, your PC asks your ISP's DNS server to resolve www.coke.com to a numeric IP address so that it can send an HTTP request. The DNS server seeks a root DNS server, which points it to a DNS server for the .com TLD, which in turn points it to a DNS server for coke.com, which at last, knows about www.coke.com and serves up the IP address.
DNS supports a feature called reverse lookup – you can ask a DNS server about an IP address, and if a record exists for it, the server will return the host name associated with that address. ISPs like AOL use reverse lookups to validate that mail servers sending email to AOL are who they say they are. Kind of a trust but verify thing to keep out the most unsophisticated of the riff-raff.
So what did I break? Back in '98 when
majikshop set up those two boxes we registered their names with our ISP (AT&T), and they stuck those two names into their DNS database. Then they delegated responsibility for reverse DNS lookups for our block of AT&T-assigned IP address to shadowrun and routerguy. As long as the aliases were there, servers out on the 'net could trace the DNS breadcrumbs to AT&T and then to us. Once I took the aliases out, I broke the chain. *Shazam* no one on the 'net could find out what host belonged to any of our addresses. ISPs like AOL stopped accepting mail from us, and a couple of days later the help desk started getting calls.It took a couple of hours of sleuthing this afternoon to dope this all out. AOL (perhaps unbelievably) gets kudos for having very responsive and knowledgeable support people. AT&T did pretty well too. For the short-term fix I put the aliases back in our DNS records. For a long-term fix I have a request in to AT&T's DNS group to change the records to the correct server names.
So,
majikshop, how's that for a lasting legacy? 