Today's task is preparing for a firewall cut-over tomorrow night. I thought that all was going well, until it rather abruptly wasn't. From an email I just sent to the Boss and several management colleagues...
Subject: FYI: What happened to email!?!?
The short answer is: I broke it, and then I fixed it.
The long answer is: Pat and I are prepping to cut most City Internet services over to our new Fibertech Internet service tomorrow evening. Part of that work is going through the configuration of the firewall that handles the existing AT&T Internet feed inch-by-inch and determining what gets moved into the new firewall and how. During my config crawl this morning I found three or four address entries relating to the mail servers and the NMS servers which both my understanding of the configuration and the firewall's own representation of the config said were unneeded duplicates. In order to make my own task of analysis and translation easier, I deleted the entries. This appears to have caused the failure of mail traffic in the past couple of hours. After Tom notified me of the problem I went into the firewall, looked over the config (which appeared to be 100% correct) and determined that the simplest course of action was to restart the firewall. Total downtime for the restart was roughly 1 min; subsequent testing has shown that mail is again moving properly (that and the influx of Spam into my inbox). Firewall stats show that it has handled > 4,000 emails since being restarted.