netcurmudgeon (netcurmudgeon) wrote,

  • Mood:
  • Music:

Twenty two and a half hours

That's how long the MS Exchange repair utility took to run through my department's email store. Starting at 4:00 PM yesterday and running straight through to 2:30 PM today. To say that the people in the department were unhappy with the lack of email (and the lack of connectivity for Blackberry users enterprise-wide) is a substantial understatement.

It goes something like this. Late yesterday morning one of our Exchange servers was infected with a worm (despite being up to date on security patches and having current McAfee anti-virus .dat files). The worm apparently modified the McAfee "do not scan" list (trying to avoid detection, most likely).

Well, modding the "do not scan" list had the effect of removing the Exchange server's database files from the exclusion list. McAfee started scanning the database files as Exchange was running, and soon enough found something it didn't like. It "cleaned" the database transaction log file for our message store and *BLAMMO* Exchange had a grande mal seizure.

My crew repaired the smaller mailbox stores in the message store first, leaving the 41 GB MHIS message store for last. By 16:00 yesterday they started eseutil on our mailbox store and waited. And waited. And waited. And waited.

Microsoft's documentation suggested that, for Exchange 2003, eseutil would run roughly an hour for each 10 GB of mailbox store. We quite reasonbly through that the tool would finish by late evening. By morning it was sitting at the last stage "deleting unicode cleanup table" (WTF? That's about as informative as "PCLOADLETTER" from Office Space)

There it sat. ALL. DAY. LONG. At the twenty-two-and-a-half hour mark *BING* it was done. Exchange mounted the mailbox store, email started flowing, and life returned to normal.

The fallout will start falling on Monday. As surely as the sun will rise.
Tags: email, exchange, virus, work

  • It's that time again...

    It's double bonus time for us right now: budget season and E-Rate season. In a time of budget crisis and ever shrinking resources, we (the IT…

  • My life...

    ...for the past three weeks, has been eaten by BTOP -- The Broadband Technology Opportunity Program grant. Hartford is applying. Time from the…

  • E-Rate, how I love thee. Let me count the ways...

    Sent by me to our E-Rate consultants today under the heading FYI... E-Rate administrative burden: Jane & Debi; Just a data point for you if you…

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.