Office Communicator “Outlook Integration Error” problems when using ISA 2006 and Exchange Kerberos Constrained Delegation

Now that’s a wordy title. I’ve been meaning to write this up since about March in more detail with some fancy diagrams, but I’ve finally given in and decided to just get the information published and update it later. One of the nicest features of ISA 2006 is the ability to use Kerberos Constrained Delegation (KCD) in reverse proxy scenarios. This can used to publish applications like Exchange or SharePoint externally and allows passing NTLM credentials over an SSL channel to ISA which authenticates to Exchange or SharePoint via Kerberos on behalf of the user, making for a seamless experience regardless of location. Read: No freaking login prompts outside the firewall, even in Outlook Anywhere with ISA performing pre-authentication of users. Jason Jones has an outstanding article already written outlining how to set up KCD with Exchange 2007 and ISA 2006 that I recommend following if you’re interested. The point of this article is not to explain how to configure KCD, but to highlight two issues you’ll find when deploying this with Microsoft Office Communicator. If you’ve deployed KCD for Outlook Anywhere and have users with Communicator outside the firewall they’ll probably see the dreaded “Outlook Integration Error” or “Communicator could not retrieve calendar or Out of Office information from Exchange Web services” warnings.

There are actually two issues here that we need to resolve. The first is that Communicator itself actually uses some slightly different RPC over HTTPs logic than Outlook which causes the KCD authentication through MOC to flat out fail. In R1 of OCS 2007 I found the clients were prompted for credentials 3 times for the Outlook integration and then the integration would fail even with correct credentials. With the R2 client you’ll no longer see the authentication prompts, but the Outlook Integration error would eventually show up. The problem is in how ISA 2006 handles POST requests than do not have a POST body, which is apparently the difference between Outlook and MOC’s logic. There is a hotfix available for this which requires running a .vbs script to make the change to ISA. You can find that hotfix and script here: http://support.microsoft.com/default.aspx?scid=kb;EN-US;942638. You actually need to apply the ISA 2006 Supportability Update package before you can install this hotfix and that package is here: http://support.microsoft.com/kb/939455/.

The second issue you’ll find is that when ISA offers Communicator the Negotiate and NTLM authenticate headers Communicator actually tries to negotiate and fails. This can be remediated by changing ISA to offer only NTLM headers to clients. There is another hotfix and .vbs script to fix this issue which you can find here: http://support.microsoft.com/kb/927265/en-us. One warning I should point out is this is a system-wide setting and will disable Kerberos for outbound-proxy scenarios. I’m not a big fan of ISA for anything other than a reverse proxy so this had no issues on my environment, but be careful to evaluate your existing rules if you use ISA for anything else.

You can test out the second hotfix without making any changes to the ISA server by going in to IE’s advanced settings on a client and unchecking the box “Enable Integrated Windows Authentication.” (Thanks to Scott Oseychik for this tip). Contrary to the outstanding verbiage, this only disables Kerberos authentication in IE and will force IE to only try to authenticate via NTLM.

Once you have all of your hotfixes installed you should be able to login to MOC only and receive no more Outlook integration errors. Perfectly seamless authentication anywhere you are. Jason Jones pointed out the two hotfixes for me originally, so a huge thanks is in order to him.