Y2DST Headaches and Microsoft Advil 2007

As most of you in the IT world have figured out, Congress’ well-meaning push to save $30-odd million a year in energy costs has ended up costing the IT industry and the economy considerably more than that in the changeover.

After spending 2 weeks patching systems and trying to make the Exchange calendar rebase tool work (unsuccessfully), I woke up Sunday morning hoping for the best and expecting the worst. As it turned out, all our Windows 2000 systems were unpatched. Apparently it was the Windows 2000 patch that was the one Microsoft wanted $4000 for – luckily they published a workaround late last week that involved patching the registry with the new timezone data.

In trying to troubleshoot this further, I’d discovered that Microsoft had set up a DST Support chatroom for anyone suffering from DST pains. I posted my problem and hunkered down to wat, as it looked like the Microsoft experts in the room were pretty buy, mostly with fairly inane questions from people who could have figured out the answer with a quick Google search a few weeks ago. After a short wait, I got a private chat request from JamesC saying he thought he could help me and would be happy to do so in a one-on-one chat. He also helped me boil down KB914387 to the simple language of “Back up your registry, patch it, run this script, and this is the backout procedure”, which allowed me to patch the Windows 2000 servers in fairly short order.

The big issue we’d been running into is that the rebasing tool kept crashing on the third mailbox when trying to generate its list of mailboxes and their associated time zones. I decided to head into the office where I had more screen space as well as some peace and quiet from the kids. As I got in, James sent me a debugger tool to install while he quickly grabbed a bite to eat.

I fired up the Microsoft-provided VM and loaded up the debugger while James ate and set up a LiveMeeting session so he could remotely control the session. This is where it got crazy. James was working from home, on his Mac, via VPN and RDP to his machine at work, from there was connecting to the LiveMeeting server, where he had remote control of my RDP session into the virtual machine running on my laptop, and from that session we frequently had another RDP session going to the mail server. I’m amazed it worked at all. I’m also amazed that Microsoft lets James have a Mac.

James spent the next several hours poring over debugger output and fiddling with assembly code trying to make the application do his bidding. Backing him up was none other than the guy who wrote the rebasing tool in the first place. After several hours of this, they both threw their hands up and resigned themselves to the fact that this approach wasn’t going to work (it must have been a weird issue if the guy who wrote it couldn’t even grok it) and that we’d have to try the manual approach. After a few false starts, James got the tool to do its thing.

Naturally, there are a few users this morning who have some appointments that are “pooched”, but that was expected.

It was truly impressive to watch James engage in Extreme Nerd Sports and poke at assembly and debug code in an effort to make the machine do his bidding. Over the course of the 8 hours, I got to know James a little. I ran across his blog (which I won’t link here to keep his personal blog from being associated with Microsoft), and discovered that he is also a committed Christian – yay for God putting the right people in the right places!

James has been spending the last few months working 16 hours a day on DST conversion. His official job at Microsoft is debugging Exchange code, and from the looks of it, he’s darned good at it. In order to help us poor customers, he’s had to miss not only Valentine’s Day but his wife’s birthday as well. I hope Microsoft makes it worth the trouble. You’ve earned yourself a serious break. Mrs. C, you are a saint. Thank you for letting us pick your hubby’s brain.