Depending on the role, Lync 2013 servers will have one or two SQL Express instances installed as part of the Lync Deployment Wizard steps. All servers deploy an instance called RTCLOCAL, and Lync Front Ends will also deploy an instance called LYNCLOCAL. Generally, there is no configuration or action required on these instances.
If you were to manually install a SQL Express instance the minimum and maximum memory values are essentially unlimited. Technically, this isn’t a realistic view because SQL Express is actually limited to 1 GB of memory, but it’s important to know the out-of-the-box defaults.
I’ve noticed that the Lync installation wizard actually configures static minimum and maximum values on each instance of both RTCLOCAL and LYNCLOCAL, and these are based on the amount of RAM presented to a server at installation time. You can see an example here, which is from a FE that uses Hyper-V dynamic memory and only had around 2.19 GB RAM available.
The instance will always float between 270 MB and 337 MB of memory, which is quite a bit below SQL Express’s ability to address up to 1 GB. We want both of these instances to always be able to address 1 GB, so this is not ideal!
Each instance uses slightly different defaults, but for the sake of documentation I’ve validated the RTCLOCAL instance will be set to a minimum of 12% and maximum of 15% memory. LYNCLOCAL will be set to a minimum of 6% and maximum of 8% memory.
If the server was installed with a minimal amount of RAM, this could negatively impact the Lync services because the SQL instances won’t enough memory available to function properly. You’ll likely be able to start services, but find they stop after a period of time and log events such as this one:
There is insufficient system memory in resource pool ‘internal’ to run this query.
Cause: The connection to the database might be broken.
It’s fairly obvious that the process needs additional memory, but simply adding memory to the server won’t resolve the issue. The “gotcha” here is if you install Lync with minimal memory for some reason and then bump it up later, Lync’s SQL Express instances won’t actually leverage the memory because of the static max/min values configured at installation time.
You can manually adjust these values within the SQL Express instance, but the Lync process will actually reset your modifications back to these percentages after a period of time.
The permanent fix is to re-run Step 1 and Step 2 in the Lync Deployment Wizard on each machine that had RAM added after the initial installation. This will set the RTCLOCAL and LYNCLOCAL instance max/min memory configuration to the percentages above based on the new amount of installed memory. Now of course, SQL Express still won’t be able to access any more than 1 GB of memory, but this fixes any potential ceiling below that number.
Big thanks to my ExtraTeam colleague, Chris Lehr, for helping track this down.