Publishing Exchange Web Services remotely only for Lync

A recent Lync project for me involved a scenario where the customer who was not publishing any aspect of Exchange remotely. They were using Good for all email to mobile devices, and had elected not to publish Outlook Anywhere or Outlook Web app remotely. This presented some challenges in terms of features available to remote Lync clients since a lot of functionality comes from Exchange Web Services (EWS). If EWS is not available a Windows Lync client can fall back to pull some data from Outlook via MAPI, but that does nothing to help a Mac Lync user, Lync mobile devices for iOS, or any Lync Phone Edition clients. All of those require EWS to be published remotely or there will be some feature loss and very visible errors within the user interface.

To work around this issue we started with a goal to publish only Exchange Web Services to support Lync clients, and still prevent all other types of clients from connecting to Exchange remotely. Publishing only EWS is a straightforward process and can be done through the TMG publishing rules. After using the Outlook Anywhere publishing rule template in TMG simply edit the paths tab and remove everything except /EWS/* and /Autodiscover/*:

image

This will effectively allow remote users to connect and do autodiscover queries. Remote Outlook clients will perform a successful autodiscover lookup, but see Outlook Anywhere is disabled (assuming you’ve disabled it on the CAS!) and not attempt a connection. You’ll also want to make sure the ExternalURL property of your EWS virtual directories are populated with the correct FQDN and that public names for that URL and autodiscover are allowed in TMG:

image

This would work great if all you had in the world were PCs, but the challenge here is the Mac Outlook and older Entourage 2008 Web Services clients are entirely based on EWS and by configuring the previous tasks in TMG we were now allowing those remote clients to connect. We needed some way to block those clients so we settled on filtering based on the User Agent string, which is a unique string within the HTTP header identifying the type of client making a request. We started with working out which user agents we actually needed to allow and came up with the following list after running some logs and tracking the user agent headers in sample client connections:

  • Lync iOS (iPad and iPhone): Microsoft Lync iPhone
  • Lync Phone Edition: OCPhone
  • Lync PC: OC
  • Lync for Mac: MC

TMG actually has a User Agent filtering tool built in, but it was very unfortunately written backwards for what we needed. You can see here that TMG is expecting you to explicitly block specific user agents that you don't want to allow:

image

That seems like a good idea if you know all the strings for every possible EWS client out there, but what happens when a new client comes out that’s not specified here? It would be allowed to connect and bypass the restrictions we were trying to impose. In this scenario a load balancer such as an F5 BigIP or A10 Networks AX device can be incredibly handy because of their iRules and aFlex engine. In our case we were able to use the aFlex rules to block requests from everything except the clients we wanted to explicitly allow. The actual code is here:

when HTTP_REQUEST {

#log local0. "[HTTP::header "User-Agent"] INBOUND"

if { ([HTTP::header "User-Agent"] matches_regex "Lync.*") or ([HTTP::header "User-Agent"] matches_regex "Microsoft\+Lync\+iPhone\/.*") or ([HTTP::header "User-Agent"] matches_regex "MC\/.*") or ([HTTP::header "User-Agent"] matches_regex "OC\/.*") or ([HTTP::header "User-Agent"] matches_regex "OCPhone.*") } {

pool ews

} else {

#log local0. "[HTTP::header "User-Agent"] REJECT"

reject

}

}

This method is not foolproof since a user agent can be spoofed fairly easily, but it met the needs of this particular project. You should also keep in mind that if Outlook Web App and the Exchange Control Panel are not published remotely then users will be unable to edit their voicemail options or manage personal call answering rules from the Internet. In my case requiring users to connect their VPN to use those features was an acceptable solution.

Lync and Exchange Web Services Over HTTP

Spoiler: It doesn't work.

The behavior you'll see is that the Lync client will issue the Autodiscover query, receive a successful response, and then never even attempt to contact the EWS URL. No DNS lookup, no HTTP request, nada. Consequently you'll see "EWS not deployed" in the configuration info or get the red bangs on the conversation history and phone tabs.

The only fix available is to modify your EWS virtual directory URLs to be HTTPS instead of HTTP, which you probably should be doing anyway, but I have run across deployments where this was not the case. After the change to HTTPS the Lync client will begin contacting the EWS URL correctly.

Lync Claims EWS Not Deployed

In the last few Lync deployments I've done I've run into two different instances where the Lync client was failing to login to Exchange Web Services to retrieve the conversation history and user voicemail. In both cases there wasn't actually the red exclamation mark on those two tabs in the UI like you'd expect if there were an error; the client just hummed along like nothing was wrong. In each scenario if I viewed the configuration information you would see the client report "EWS Not Deployed", which was odd because Exchange 2010 was most definitely deployed at both customer sites.

Sidenote: The EWS polling takes roughly 30 seconds to reach this state. If you view the configuration info immediately you'll see "EWS OK", which is only because Lync has tried yet. So be careful when testing this and thinking everything is just fine.

Solution 1: Verify the InternalURL and ExternalURL for the Web Services virtual directory are entered
The first fix was incredibly easy and after some more digging we determined this was only occurring when a client was external and logging in through an Edge server. When we looked at the Exchange Client Access Server we found this customer had not actually entered an ExternalURL parameter for the Web Services virtual directory. This works just fine for Outlook clients, but Lync is expecting this value to be filled out. If it's not entered it assumes EWS is not deployed externally and doesn't attempt a connection, which is a pretty reasonable action. You might argue the Outlook action is incorrect and it should treat it the same way. But anyway, the fix is to just fill out the ExternalURL and Lync will begin using that value to login to EWS successfully.

Sidenote 2: The information discovered by Lync via Autodiscover is cached in the registry at HKCU\Software\Microsoft\Communicator\<SIP URI>\Autodiscovery (Can you tell a Lync dev wrote the regkey name? Autodiscovery instead of Autodiscover?) You'll see entries for the internal and external URLs for the Availability Service, Exchange Control Panel, Exchange Web Services, and Out of Office Assistant. I've been able to delete this entire registry key for quick testing and found it recreated with no issues.

Solution 2: Place https://<Your SMTP domain>/ in the Local Intranet Zone
The second instance of this issue was a little more complicated, and still doesn't make much sense to me, but I figured I would share. In this case the customer did not have Outlook Anywhere published so we expected it to fail externally, but this error was actually occurring internally. After verifying the InternalURL was filled out correctly we started doing some traces and noticed the Lync client would make a GET request to the /Autodiscover/Autodiscover.xml file on Exchange, Exchange would return a 401 Unauthorized challenging for credentials like we expected, and then the trace died. There were be no more responses from the Lync client IP address sent to Exchange in the logs. We verified this on multiple machines and operating systems and concluded that the Lync client would never respond to the credential request! For what it's worth, Autodiscover was working fine for Outlook clients and no special configuration had been done to Exchange.

So we put a call into PSS and they told me Lync will not read the SCP for Autodiscover in AD, even if the Lync client is internal and that it will do its own Autodiscover lookup (Can anyone confirm/deny this?). Therefore, it will fall back to https://domain.com/Autodiscover/Autodiscover.xml, and if that fails it should move on to https://autodiscover.domain.com/Autodiscover/Autodiscover.xml like an Outlook client. This is where it got weird - PSS told me from the ETL trace Lync was not falling back to the 2nd option, yet I could clearly see it make a request to IIS and not respond. From what they saw the Lync client was getting stuck on the 1st option which didn't really exist. In any event, they had me add http://<domain.com>/ to the Local Intranet Zone on the client. Even though we knew this was not the location of Autodiscover and I really didn't think it would make a difference it did solve the problem. After adding entry this we saw clients then try to resolve autodiscover.domain.com and grab the Autodiscover.xml file correctly from https://autodiscover.domain.com/Autodiscover/Autodiscover.xml. At this point the EWS status in the configuration information returned to EWS OK.

Sidenote 3: There is a thread on the Technet forums about this issue which suggests editing your applicationhost.config file on the Exchange server. I have to recommend against this and as you can see in the comments it hasn't really fixed the problem for anyone. The solution is more likely one of the ones presented here.