A few months back I was involved in a discussion about what the subject name of an OCS Edge Server's A/V authentication certificate should be. Some folks were saying to use the Edge server's internal FQDN and others were saying to use the external, public FQDN you define for A/V. I was in the camp using the external name, but the odd thing was both sides said their approach worked. There is definitely some confusion about what name you should use and Microsoft has actually published directly conflicting information which further confuses the issue. Some testing I've recently done clears up why so many documents and people contradict each other - the subject name doesn't matter. Really. You could put whatever you want in that subject name, assign it to A/V authentication and it will work flawlessly. The purpose of this certificate per the Technet documentation:
The private key of the A/V authentication certificate is used to generate authentication credentials.
Specifically, it's not used for encryption or MTLS even if that's not made clear anywhere. Let's take a step back and clarify a few things for some background:
- There are two services that run on the Edge server with "A/V" in the name. If you’re not familiar with the difference, Jeff Schertz’s More on OCS Edge Server Certificates article has a good explanation for some background on what the difference is between the A/V Authentication and A/V Edge services, but basically - the A/V Authentication service is internal facing and A/V Edge Service is external facing.
- There is no certificate assigned to the A/V Edge service because encryption for external A/V traffic is provided by SRTP.
- The certificate for A/V Authentication is only used by internal clients when trying to communicate with an external or federated client. This means you can (and should) use an internal certificate authority to issue this certificate. There is no benefit or need to use a public certificate for A/V authentication.
Let's walk through a little example here as if I was trying to figure out what name to use for my A/V authentication certificate. I have the following environment:
- Public Domain: confusedamused.com
- Internal AD Domain: ptown.local
- SIP Domain: confusedamused.com
- Edge Server Internal FQDN: edge.ptown.local
- A/V Edge Service FQDN: av.confusedamused.com
So with that information what should I use as the certificate name for the A/V authentication certificate? If you consult the Technet documentation topic Set up Certificates for A/V Authentication you’ll find this note (emphasis is mine):
The subject name should match the fully qualified domain name (FQDN) of the A/V Edge Service published by the external firewall, or the FQDN of the VIP used by the A/V Edge Service array on the external load balancer (that is, if the Edge Servers are load balanced).
So based on that blurb, my A/V authentication certificate subject name should be av.confusedamused.com. Fair enough.
I ran through the OCS 2007 R2 Edge Planning Tool for a sanity check. You can see the result below, but the tool follows the Technet documentation and uses the external FQDN I defined for the A/V Edge Service when it asked.
A group of MVPs and Microsoft employees published a document called Deploying Certificates in Office Communications Server 2007 which says the following about the A/V authentication certificate (emphasis is mine again):
Must be the FQDN of Audio/Video authentication server in DNS.
Well that calls out the name of the authentication server, not the A/V Edge Service. I think this comes down to really just poor wording in the document which contributes to confusion, but what is the name of our A/V Authentication server? It would be the same name as the internal Edge interface, right? The A/V Authentication server is the Edge server, not the external FQDN. So now we're being told to use the internal FQDN, edge.ptown.local as the subject name.
Also released by Microsoft was a document called OCS 2007 R2 Walkthrough - Scale to Load Balanced Edge Server which completely contradicts Technet and the Edge Planning Tool (emphasis mine):
- Access Edge Internal (Corporate Certificate). In our sample topology, the subject name would be set to ocsedge.contoso.com, the FQDN of the Edge Server internal interface.
- A/V Authentication Internal (Corporate Certificate). In our sample topology, the subject name would be set to ocsedge.contoso.com, the FQDN of the Edge Server internal interface.
This seems to match up with the certificates document and is somewhat backed by the exact same Technet article I referenced earlier which says:
As a security precaution, you should not use the same certificate for A/V authentication that you use for the internal interface of the Edge Server.
This begs the question "Why would I ever even try to use the same certificate?" The only logical reason would be perhaps because they use the same subject name. That jives with the Scale to a Load Balanced Edge Server documentation. If we're thinking about this in terms of MTLS connections, you would have to think that this makes the most sense. In your OCS Forest properties if you added an A/V Edge server with the name edge.ptown.local for port 5062, it's reasonable that you'd expect the A/V Authentication service operating on port 5062 of the internal interface to offer a certificate matching this name. If it presented something wrong, say maybe the external FQDN of the A/V Edge service it should fail, right?
Well, the truth is the name doesn't matter. There isn't MTLS validation happening on port 5062 the same way you'd expect MTLS between servers on 5061. I think the reason the certificate requirement issue hasn't been pointed out yet is because it's never caused a problem - it works either way. I can use a certificate with a subject name gobblygook.confusedamused.com and media relay authentication through the Edge server works just fine. It just needs a certificate to generate authentication credentials for the media relay process. Go ahead and try it out - put whatever name you want on the certificate and it will still work.
So while the subject name doesn't really matter, if you're still interested in adhering to best practices I would recommend using the external facing, public A/V Edge name. In the example earlier this would be av.confusedamused.com. Hopefully Microsoft will update the certificate and scaling documents with a clarification and make them more consistent with the rest of Technet.