The past few days I spent wrestling an address book server issue in OCS and I wanted to share the solution. The quick version: make sure you have your server side and client side hotfix revisions match up.

If you want the whole story…the specific details of this case involved a Front-End server which had the OCS 2007 R2 April hotfixes, but the MOC clients had the July hotfixes applied. The issue first manifested itself with clients reporting ABS damaged:

Communicator cannot synchronize with the corporate address book because the corporate address book file appears to be damaged. Contact your system administrator with this information.

We resolved this by deleting the entire contents of the address book file share and forcing a resync of the address book. We also deleted GalContacts.db from a few user workstations, but later found the client error actually disappeared on its own without removing the file.

Things hummed along nicely for a week or so until a large amount of users (600+) were enabled for OCS one Friday evening. The following Monday previously enabled pilot users were reporting they still didn’t have a SIP URI in their address books for the mass-enabled users. The GalContacts.db file was also still showing a timestamp from the day the mass change occurred, indicating they had not downloaded an update yet.

We took a peek at the Front-End logs and it appeared to be generating address book files correctly. The odd thing was in looking at the IIS logs we actually saw quite a few 404 errors of MOC clients trying to request delta files that did not exist. Other users showed successful downloads of the latest delta files which should have included the changes, but they weren’t being applied to their local GalContacts.db for some reason. I also saw those same clients registering a success end up using the fallback logic and downloading older address book files even though they had the newer versions. Very, very strange. Any client we deleted GalContacts.db on would pull down the latest full address book with no issues. The clients looking for deltas that didn’t exist we probably caused by deleting the address book files previously.

Side tip when looking at the IIS logs: Full files start with F-xxxx and delta files follow a D-xxxx-xxxx naming convention. Also, .lsabs files are used by MOC while .dabs files are used by Tanjay devices.

At that point I noticed the mismatch in server (April hotfixes) vs. client (July hotfixes) versions and suggested we get the latest fixes installed on all sides. While that suggestion made its way through change control procedures we opened a PSS case with Microsoft to hit the problem from another angle. The engineer we spoke with immediately blurted out that we needed to match the hotfix versions as soon as we described the behavior. It sounded to me like this was one he had heard before or was familiar with so while we didn’t have a second approach this definitely helped accelerate the change control ticket to an authorized state. After we fully patched the Front-End using the new ServerUpdateInstaller (a lifesaver), applied the back-end database hotfix, and installed the client October hotfix the address book went back to functioning properly. There were a couple of users that needed to delete the GalContacts.db before everything went back to normal, but most of them picked it up without intervention.

As for root cause, the KB 972403 article actually does reference applying both the MOC and server fix together, but the July server hotfix document doesn’t describe this behavior or even mention it. Personally, I think the underlying issue was having the 3.5.6907.37 hotfix on clients while the abserver.exe file was still at 3.5.6907.0. In any case, I learned a lot more about the ABS than I ever cared to, but it was great information that will surely help in the future.

This one killed me today. Exchange 2007 SP1, with Rollup Update 6 on Server 2008. Everything working perfectly with one exception – the offline address book (OAB) never downloads from the file distribution point for Outlook 2007 clients. Works fine via public folders, but not web-based. No error, no timeout, no progress indicator, no login prompt, Outlook just looks like it’s endlessly trying to download the OAB. I double-checked all the URLs, flipped around SSL settings, but still couldn’t figure out why it wouldn’t download. I would have been happy to see an error so I had something to search on. There were actually 2 problems here that made the situation a real pain in the ass.

First – the same bug that affects Outlook Anywhere on Server 2008 apparently does a number on the OAB too. The solution is to turn off kernel-mode authentication in IIS. Run this command to fix that issue and you’re halfway there. I ran across some blog that mentioned Rollup Update 7 may include this change by default.

C:\Windows\system32\inetsrv\appcmd.exe set config /section:system.webServer/security/authentication/windowsAuthentication /useKernelMode:false

Second – I had enabled a redirect at the Default Web Site root to dump clients to the /owa folder gracefully using the Microsoft methodology at Technet. If you read the procedure you’ll notice setting the redirect at the root sets the same redirect on every single virtual directory. So, you need to go in to each virtual directory and undo the change you made for the root. This works fine, or appears to until your Outlook 2007 client tries to download the OAB and hangs forever.

I brightly plugged the URL to the OAB.XML file into IE and was greeted with a 500 – Internal Server Error message without an authentication prompt. That didn’t seem right. After some searching I realized the reason why Outlook hangs forever is that it tries to hit this URL, gets denied, uses some back-off logic, and tries again. I believe the back-off gets longer and longer each time it fails.

What happens is that when you disable that redirect for the OAB virtual directory IIS 7 generates a web.config file in the C:\Program Files\Microsoft\Exchange Server\ClientAccess\OAB folder. This seems logical, as it overrides the redirect at the root level, and is necessary. Unlike every other web.config that is generated in the other folders like Autodiscover and OWA, Authenticated Users do not have read access to the file. This is why Outlook and IE can’t even access the /OAB virtual directory.

The fix is pretty easy. Open the web.config in the OAB folder, and give Authenticated Users both the read and read and execute permissions. Run a iisreset /noforce on the CAS server to bounce IIS. Just for good measure, on the client side I wiped out the Outlook profile, and the contents of the %USERPROFILE%\Local Settings\Application Data\Microsoft\Outlook folder. Once I recreated the profile the OAB downloaded just fine. All in a day’s fun…

This task has always been more of a pain that it ever should have, regardless of application. After trying a few of the usual hacks like requiring SSL and using a custom error page or an HTTP to HTTPS module I found I still wasn’t having any luck. From what I can tell this is because there actually isn’t any kind of default web page in the CWA virtual directory so when you browse to the HTTP version of the site you actually get a 404 “Page not Found” error before anything else happens.

I ended up keying off that idea and changed the 404 error page to be a redirect to the HTTPS page. I’m still testing this out, but I haven’t run into any issues yet with this approach. To change your site the same way:

1. Open IIS 7 Manager.
2. Click on the CWA virtual web site you want to redirect.
3. Double-click on Error Pages.
4. Highlight 404 and press Edit in the right pane.
5. Select the Respond with a 302 redirect, enter https://My-CWA-URL and click OK.
6. Run a iisreset /noforce for good measure.

I’m curious how this works for everyone and if you see any issues with this method.