I was given the task of deploying a full-blown on-premise deployment of Skype for Business Server 2015 for a small company. The deployment also included an Edge Server and a Reverse Proxy server (IIS with ARR). In this blog post I’ll discuss the deployment process in general, and also the problems (and solutions) that were discovered during/after the deployment. I’ll end the post with some check-up/misc. information.
Planning
General information about planning for Skype for Business Server 2015 can be found at:
https://technet.microsoft.com/en-us/library/dn951427.aspx
However, as I’ve deployed Lync Edge plus Reverse proxy servers before, much of the planning/homework was already done. I’ve also previously written a blog post titled “Test Lab Guide: Windows Server 2016 with Integrated Exchange 2016, SfB Server 2015 and SharePoint 2016”, where I deployed a front-end server for Skype for Business Server 2015. That said, use these two blog posts as a compliment for this guide – they include some deeper information not discussed here. Front-end DNS entries haven’t been discussed earlier however, so here goes:
Front-end DNS
I used the following entries:
A-record:
fe.domain.com
C-names:
lyncdiscoverinternal.domain.com
sip.domain.com
meet.domain.com
dialin.domain.com
webext.domain.com
The next chapter will present the actual front-end installation, including some dilemmas/gotchas. After that I’ll continue with the Edge and Reverse Proxy installation.
Front-end installation
For the front-end installation I followed TechNet’s deployment article with some twists. As usual, it didn’t quite work as planned 🙂 I’ll now try to explain the process in detail, including some problems along the way.
Before starting the actual setup, I double checked that my network and AD infrastructure was setup correctly. I also checked that I had a working ADCS (not setup by me), so I could get internal certificates assigned on the front-end. (This actually turned into quite a mess, see the next chapter about front-end certificate gotchas). Finally I checked that I had created/requested all the DNS aliases needed in the internal DNS zone.
With all these steps done, it was time to move along to the actual front-end installation. My steps:
-
Installed pre-requisites for Windows Server 2016, which are slightly different compared to Windows server 2012 R2. Information can be found at: https://blogs.perficient.com/microsoft/2017/08/skype-for-business-how-to-install-on-windows-server-2016/ for example. (The issue regarding IIS URL Rewrite module wasn’t a problem in my deployment, btw).
-
Add-WindowsFeature RSAT-ADDS, Web-Server, Web-Static-Content, Web-Default-Doc, Web-Http-Errors, Web-Asp-Net, Web-Net-Ext, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Http-Logging, Web-Log-Libraries, Web-Request-Monitor, Web-Http-Tracing, Web-Basic-Auth, Web-Windows-Auth, Web-Client-Auth, Web-Filtering, Web-Stat-Compression, Web-Dyn-Compression, NET-WCF-HTTP-Activation45, Web-Asp-Net45, Web-Mgmt-Tools, Web-Scripting-Tools, Web-Mgmt-Compat, Telnet-Client, Windows-Identity-Foundation
-
-
Installed Silverlight
-
Continued with “official” documentation from https://technet.microsoft.com/en-us/library/dn933889.aspx?f=255&MSPPError=-2147217396
-
Created the file share (local).
- Important:
“The file share cannot be located on the Enterprise Edition Front End Server, but can be located on a Standard Edition server.
You can define the file share in Topology Builder before you create the file share, but you must create the file share in the defined location you define before you publish the topology”.
- Important:
-
-
- Installed administrative tools.
- Prepared Active Directory, using an AD account with enterprise/schema admin rights.
- Schema Preparation went fine, but forest preparation failed:
-
-
- I hadn’t seen these errors before. Some googling led me to:
-
- Still no luck however:
-
-
-
-
- What now? Sigh.
- More googling:
-
-
-
-
-
-
-
- Success! The -GlobalCatalog option was the key.
-
- I finally moved over to preparing the domain. The domain preparation worked without problems for a change 🙂
- I then created and published a new topology, again following https://technet.microsoft.com/en-us/library/dn933913.aspx
- Nothing fancy, just the basics with a standard edition front-end and without all the bells and whistles. (Edge server will be configured later):
-
-
-
-
-
- Media server is collocated.
- Web services external url is specified as webext.domain.com (in the DNS front-end information earlier).
- Simple urls are also specified according to the DNS information.
- Time to publish the topology…
- But wait. Read carefully: “If this is Standard Edition, you will need to run thePrepare First Standard Edition Server process from the Deployment Wizardbefore you publish a topology. This prepares for Standard Edition by installing a SQL Server Express Edition instance and creating the Central Management Store”.
- Publishing will actually fail if you forget to prepare first! Been there, done that 🙂
- So, new order is:
- Prepare
- Publish
- Done! …or not 😦 SIGH! New errors:
-
-
-
-
-
- Getting sick of all the (never-before-seen) errors. Much googling led me to the following commands:
-
-
-
-
-
-
- I got no source for the above (too much googling), and afaik everything even looked OK in the screenshot.
- Even more googling led me to:
- https://social.technet.microsoft.com/Forums/ie/en-US/b842c139-9cde-4889-9a24-eb09e3cd0064/automatic-collection-of-data-failed-skype-for-business-server-cms-isnt-in-active-directory?forum=sfbfr
http://www.zerohoursleep.com/2013/04/error-when-installing-lync-server-2013-local-configuration-store-cannot-find-any-global-catalogs-in-forest/ - From here on I’m quite clueless what I did. Was it some random PowerShell”magic”, or was it a server reboot that finally did the trick? (I assume the latter…)
- However, by some voodoo magic the publishing wizard completed without errors after the server reboot:
-
-
-
-
-
-
- (Persistent chat and Edge will be enabled later on)
-
-
- Now it was finally time to install the system components, following https://technet.microsoft.com/en-us/library/dn951417.aspx?f=255&MSPPError=-2147217396
- Make a note of the following: “Before you follow these steps, make sure you’re logged onto the server with a domain user account that’s both a local administrator and a member of the RTCUniversalServerAdmins group”.
- Ran Step 1, Install Local Configuration Store. No problems.
- Ran Step 2, Setup or Remove Skype for Business Server Components. No problems.
- Ran Step 3, Request, Install or Assign Certificates.
- I checked with the “AD CS guy” that everything was in order for the request. It was.
- I made a custom request with the private key marked as exportable.
- I assigned the certificate (both for default and OAuth).
- (NOTE! Now using a public certificate, read about the gotchas in next chapter)
- It was now time to start all the SfB services.
- This is done from SfB management shell with the command “Start-CsWindowsService”
- All good.
- I finally configured integration between Exchange and Skype for Business (using OAuth). I wrote how to do this in my previous blog post, so have a look there for details.
- I also enabled MAPI for the company’s Exchange server for a smoother integration between SfB and Exchange. I’ve also written about this procedure in a previous blog post.
- As a last step, I wanted to bulk-enable a bunch of users from an AD group.
- I found a nice script from https://gallery.technet.microsoft.com/office/Bulk-Enable-Users-from-an-9a982357
- Ran it:
-
- Checked the result from SfB Control Panel:
-
- Success! 🙂
- Finally tested a SfB client, everything seemed OK!
Front-end certificate gotchas
Certificate requirements for internal servers in Lync Server 2013 (same for SfB Server 2015) can be found at:
https://technet.microsoft.com/en-us/library/gg398094%28v=ocs.15%29.aspx?f=255&MSPPError=-2147217396
Some notes:
“Although an internal enterprise certification authority (CA) isrecommended for internal servers, you can also use a public CA. For a list of public CAs that provide certificates that comply with specific requirements for unified communications (UC) certificates and have partnered with Microsoft to ensure they work with the Lync Server Certificate Wizard, see article Microsoft Knowledge Base 929395, “Unified Communications Certificate Partners for Exchange Server and for Communications Server,” at https://go.microsoft.com/fwlink/p/?linkId=202834.”
Source: https://technet.microsoft.com/en-us/library/gg398094%28v=ocs.15%29.aspx?f=255&MSPPError=-2147217396
“For internal servers (e.g. Front End Servers and Edge Server internal interface), you can request the certificates from your internal CA. For Reverse Proxy and Edge Server external interface, you should have public certificates installed.You can use SAN certificates, for example.”
“Although this approach technically can work it’s not recommended as (1) it’s not best-practice to use a third party certificate on the internal Front End servers and (2) it’s not advisable to publish your internal server FQDNs and general namespace into an external certificate.”
The above information really begged for internal certificates on the front-end. And yes, I tried the approach. Oh boy, DID I. MANY, MANY times. It always resulted in a mess however. Because of this, we’re now using a public certificate on the front-end. I’ve later realized that I COULD have made it work with internal certificates also. At least to some extent. HOWEVER, you get VERY tired of trying something that works in theory, but not in practice. Let me list some known “problems” (not so easily “found”, at least not in my opinion):
- using internal certificates on the front-end combined with an Edge server requires SSL offloading on the reverse proxy. This is not that well documented afaik, but kind of logical. Not that logical when you have 100 other things to worry/think about though. See: http://www.uclabs.blog/2014/08/lync-2013-reverse-proxy-solution-with.html. I fought with the ARR rules quite a bit so the SSL offloading part wasn’t the hiccup for me. A bigger issue for me was the fact that:
- using internal certificates gave a certificate warning in ALL browsers except Internet Explorer / Microsoft Edge. I don’t know of ANY workaround for this, as I can’t make the internal CA root certificates available for Firefox or Chrome. Short version story: a meeting link (https://meet.domain.com/whatever/XXXX) give certificate error no matter what I do on non-MS browsers. It does not matter if the connection is made internally or via the reverse proxy.
-
The SfB / Lync CLIENT is probably happy with the internal certificates (also through IIS ARR with SSL offloading), HOWEVER I don’t see this as a 100% working solution when an external participant could be using the web client for instance (and receiving a certificate error).
I’ve written more about the certificate dilemma in a later chapter also, see “Problems after the deployment / Certificate problem revisited”.
Edge Server installation
As previously mentioned, I’ve blogged about this before in my blog post Adding Edge and Reverse Proxy Servers to an Existing Lync 2013 Environment. There’s absolutely no need to rewrite this information, as it’s completely usable with Skype for Business also. A little recap is always in place though:
- First, have a look at your network interfaces:
All looks good! Three external interfaces and one internal (all renamed). The third one in the list is the internal NIC and it has no gateway specified. (Never mind the blurs 🙂 )
-
Then add the routes needed for the internal traffic:
route add -p 1.2.3.0 mask 255.255.240.0 1.2.4.40
route add -p 1.2.5.0 mask 255.255.240.0 1.2.4.40
These are just example values. Subnet 1.2.3.0 and subnet 1.2.5.0 should use the internal NIC/interface and the gateway configured on that NIC (1.2.4.40 in the above example).
(You should also edit the hosts-file on the Reverse proxy, NOT on the edge).
- You can read about DNS and certificate requirements in my previous blog post. Please do.
- Check the primary DNS suffix. It should match the suffix from the front-end. This is actually quite an important step so don’t forget to check it!
- Manually install the internal and external certificates via MMC. Your own method may be different.
- Also export (from the front-end) and import the internal CA root certificates. This is needed because the edge server is not domain joined. Google if you don’t know how, or read my previous blog post.
“Activating” the Edge service…
The Edge service is activated on the Front-end. I like the guide I’ve used before, namely https://www.sherweb.com/blog/installing-lync-2013-edge-server/. It’s still usable for SfB Server 2015, but feel free to use whatever guide you like. Some notes from the setup:
- DO make sure that you create the new Edge Pool under the “Skype for Business Server 2015” branch, not under Lync Server 2013.
- This pool has one server
- Did not enable federation
- NOT using a single FQDN and IP for the Edge services
- Using IPv4 and no NAT.
- Specified the external FQDN the same way as in my previous blog post
- …and so forth
- Published the topology
- Exported the configuration
Moving over to the Edge server…
Same prerequisites as on the font-end:
Add-WindowsFeature RSAT-ADDS, Web-Server, Web-Static-Content, Web-Default-Doc, Web-Http-Errors, Web-Asp-Net, Web-Net-Ext, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Http-Logging, Web-Log-Libraries, Web-Request-Monitor, Web-Http-Tracing, Web-Basic-Auth, Web-Windows-Auth, Web-Client-Auth, Web-Filtering, Web-Stat-Compression, Web-Dyn-Compression, NET-WCF-HTTP-Activation45, Web-Asp-Net45, Web-Mgmt-Tools, Web-Scripting-Tools, Web-Mgmt-Compat, Telnet-Client, Windows-Identity-Foundation
- Ran setup.exe from the install media and continued following Step 5 from the guide at https://www.sherweb.com/blog/installing-lync-2013-edge-server/
- Certificates were already installed via MMC so no need to import, just assign.
- Started services (Start-CsWindowsService)
- Created DNS SRV records (Step 8). The first one is needed, so don’t try without it. It won’t work.
- Step 9 in the guide is falling a little bit short. There’s no mention about the Access Edge Configuration – you should also check that tab. See screenshot below:
- While you’re at it, have a look at the conferencing policy at the same time:
Be sure to enable needed rights for the anonymous users.
- Happy days, now moving over to the reverse proxy server installation.
Reverse Proxy server installation
I’ll just do a mini-recap as this installation is almost identical to the existing one in my blog post Adding Edge and Reverse Proxy Servers to an Existing Lync 2013 Environment.
- Checked the network interfaces (renamed and specified correct IP’s).
- Defined some hosts in the host-file. Check my above blog post.
- Installed the certificates via MMC. More detailed information in my previous post.
- Some newer information for Windows Server 2016:
- For IIS, just install the basics:
- Install-WindowsFeature -Name Web-Server, Web-Mgmt-Tools
- For ARR, download the Microsoft Web Platform installer, and from there choose to install ARR 3.0. Example: https://www.youtube.com/watch?v=ILOC91lAyBE
- Add the correct certificates to the https bindings in IIS
- For IIS, just install the basics:
- I previously followed http://jackstromberg.com/2014/11/tutorial-deploying-a-reverse-proxy-for-lync-server-2013/ for the IIS ARR rewrite rules. For some reason they didn’t work this time. Well, I’m not surprised, as IIS ARR is VERY picky about the rules. (It can actually be quite a headache to configure these).
Some alternative links for IIS ARR rewrite rules:
https://lucavitali.wordpress.com/2017/06/30/arr-how-to-setup-and-use-with-multiple-lyncsfb-sip-domains/
http://www.uclabs.blog/2014/08/lync-2013-reverse-proxy-solution-with.html
http://www.ucguys.com/2014/08/using-iis-arr-30-on-windows-server-2012r2-as-a-reverse-proxy-for-lync-server-2013.html
I used the first link as a guidance for my (new) rewrite rules. I tried many different combinations before I found this one, but to no avail. MANY thanks to Luca Vitali for something that actually works 🙂
I love the fact that you get away with using just ONE server farm/rule for all the simple urls. (We’re not using Office Web Apps).
The rule that work:
Pattern: (.*)
{HTTPS} on
{HTTP_HOST} = (meet|dialin|webext|lyncdiscover).domain.com
As a side note I noticed that you can use multiple SIP domains with this same “one liner”. In that case you just change the line to:
{HTTP_HOST} = (meet|dialin|webext|lyncdiscover).domain.com | (meet|dialin|webext|lyncdiscover).domain2.com
“|” stands for “or” and is the key here. This is an alternative approach to Luca’s implementation. See screenshot below:
I tried this with two different SIP domains, and both domains proxied nicely through IIS ARR with the above change 🙂
Problems after the deployment
All of the above would seem like the perfect deployment, right? Wrong. After some testing you’ll probably notice some problems. My deployment was no exception. When I started the SfB client (after finishing the Edge deployment), I was greeted with the following:
A quick look in SfB Control Panel and event viewer told me something was wrong with the replication:
On the Front-end:
On the Edge server:
These error messages were new to me but trusty ol’ Google led me to:
https://social.technet.microsoft.com/wiki/contents/articles/7459.troubleshooting-limited-external-calling-in-lync-client.aspx
http://communicationsknowledge.blogspot.fi/2012/06/limited-external-calling.html
http://www.wavecoreit.com/blog/serverconfig/file-transfer-agent-cannot-send-replication-data-to-replica-replicator-on-edge/
https://support.microsoft.com/en-us/help/2759117/lync-server-central-management-store-replication-failures
It didn’t take me long to realize that this was a firewall problem. I thought I had notified the firewall guy about all needed (open) ports, but this one seemed to have slipped. Well, no problem. We opened up the replication port (4443) in the internal firewall, and the replication started working again almost instantly.
To double check that the problem wasn’t with the replication service, I also used netstat and telnet:
On the Front-end:
On the Edge server:
As seen in the screenshots, port 4443 is listening as it should be. This clearly shows that the service is listening, and it shouldn’t be the cause for the problem.
I could NOT telnet to the port however, which implies a firewall problem. After fixing the (hw) firewall and starting the SfB client again, the error was gone:
Warning gone, happy days! 🙂
Certificate problem revisited
As stated in the chapter “Front-end certificate gotchas”, I had problems with the internal certificate. To illustrate the dilemma, let me show you some screenshots:
From a domain joined client:
To the left we have Firefox. As you can see, the connection is NOT trusted even though I’m testing from a domain joined client.
To the right we have Internet Explorer. The connection IS trusted (root CA is trusted).
Same thing from an external client:
“Since Lync simple URL publishing does not require any SSL Offloading if you have the External Web Site in Lync assigned with a Public Certificate, you do not need a certificate installed on the IIS.
Most likely you have assigned a private certificate from your internal Certificate Authority, and you have to assign the IIS ARR an public certificate and reencrypt the traffic for internal use.
Be aware of two point here:
- this is called SSL Offloading and requires some extra CPU load on your server
- IIS must not be “domain joined” and therefore you need to have the internal Certificate authority Root Certificates assigned as TRUSTED!”
Source: http://www.uclabs.blog/2014/08/lync-2013-reverse-proxy-solution-with.html
Well, I could probably have gotten rid of the Internet Explorer error on the external client (while still using internal certificates) by using SSL offloading on the reverse proxy. It doesn’t solve the problem with other browsers however, afaik. If you have a nice solution to this whole certificate-dilemma, please comment!
All in all I think using a public certificate on the Front-end is easier, even though you’re “going against” best practices to some extent. At least it makes your life a whole lot easier.
Final check-up’s
Skype for Business Configuration Information
You should always check the Skype for Business Configuration Information from the SfB client after a successful deployment. Press the “ctrl” key while right clicking on the Skype for Business icon in the taskbar (down right). Then choose “Configuration Information”. From here, check that everything looks OK:
- MRAS list the connection to the edge server, and it looks OK.
- GAL search tells SfB to look for the address book from the Global Address List (Exchange) instead from the internal SfB server Address Book.
- EWS Internal URL is the connection towards Exchange.
- Contact List Provider UCS (Unified Contact Store) shows that the Address Book is situated on Exchange – we’re not using a local SfB server Address Book.
- UCS (see above) Connectivity State says that the connection to Exchange is active as it should be.
- MAPI Information states that MAPI status is OK (which it should be now that I’ve configured Exchange for MAPI).
- EWS (Exchange Web Services) is also OK.
Get-DnsAndIP PowerShell script
I found an awesome script to get an overview over deployed DNS records in a Skype for Business deployment. This is a lifesaver if your head is about to explode with all the DNS/certificate/firewall/whatever information 🙂 Get the script from:
https://gallery.technet.microsoft.com/office/Get-DnsAndIPPS1-List-your-a4689878
This script (plus Snooper) actually helped me find a missing DNS SRV record. Phew. A screenshot from the script in action:
There are some “false positives” here. Only thing worth mentioning is the missing SRV record at the end. The test was also done with dual SIP domains, which is not the setup anymore. I’ve not added DNS records for federation either, as we’re not federating with any partners (yet).
Lastly, I’m NOT using lyncdiscover.domain.com in the internal DNS (intentionally). I have an explanation for this in my previous blog post Adding Edge and Reverse Proxy Servers to an Existing Lync 2013 Environment.
Snooper
Also worth mentioning is Snooper, which is part of Skype for Business Server 2015 Debugging Tools. It’s downloadable from:
https://www.microsoft.com/en-us/download/details.aspx?id=47263
It is very awesome in debugging client (connection) problems. I used it in combination with the above PowerShell script to find out that my DNS SRV record was missing for example. Here’s a screenshot from that particular “investigation”:
For some quick information on how to use snooper, have a look at http://lynclead.com/?p=148for example. The log files for the newest SfB client are located in C:\Users\xxxxx\AppData\Local\Microsoft\Office\16.0\Lync\Tracing btw.
Misc.
I’m also using the ABS Configuration Tool in this deployment to customize the Address Book. Detailed information about this can be found in my blog post Disabled (AD) users still searchable in Lync/SfB. The download link is available at: https://technet.microsoft.com/en-us/library/jj945604.aspx
SSL hardening of the SfB servers were done with IIS Crypto, https://www.nartac.com/Products/IISCrypto. I used the “best practices” template and received an “A” grade on Qualys SSL Labs Server Test, https://www.ssllabs.com/ssltest/. If you’re interested in getting an A+, have a look at https://scotthelme.co.uk/getting-an-a-on-the-qualys-ssl-test-windows-edition/ for example.
This quite much summarizes the whole Skype for Business deployment. Be sure to check my other blog post about Edge and reverse proxy deployment also, as it includes some deeper information not mentioned here.
Advertisements
Adding Edge and Reverse Proxy Servers to an Existing Lync 2013 Environment
Update: Also have a look at my newer blog post: Deploying Skype for Business 2015, including Edge and Reverse Proxy Servers
My recent task was to expand our existing Lync environment (Lync Server 2013 Standard) with an Edge and a Reverse proxy server. (This guide probably works for Skype for Business as well). Our old Lync environment had been in test usage for a while (with a rather small test-user group), but with more and more Lync users adding up it was time to expand. As a matter of fact, a Lync environment without Edge and reverse proxy servers is rather useless – you are unable to organize external meetings.
First, let me start off by saying that there are A LOT of “moving parts” involved in configuring a reverse proxy and an Edge server. You must plan for IP addresses, DMZ settings, DNS settings, certificates, firewall settings and so forth. To get a grasp of the whole picture I’m suggesting that you read/watch the following:
https://www.youtube.com/watch?v=2TdoVb1Grak
https://technet.microsoft.com/en-us/library/dn594589.aspx – A very good protocol poster (Skype for Business Server 2015 Protocol Workloads) that helps with the overall picture. It’s also very good for checking firewall requirements/port configurations. Yes, it’s a bit overwhelming but very good in the end 🙂
https://technet.microsoft.com/en-us/library/jj204756%28v=ocs.15%29.aspx – Explains (firewall) ports.
A good place to continue after this would be http://blog.schertz.name/2012/07/lync-edge-server-best-practices/. It’s a very nice all-around document about Edge and Reverse proxy. Pay close attention to the chapter about Best Practices. Our goal in the end was to get something that resembles this picture:
Fig 1. Lync Front-end in combination with Edge and Reverse proxy – Simple Topology. (Pic source: http://blog.schertz.name/2012/07/lync-edge-server-best-practices/)
Even after a lot of reading, It’s hard to know where to start (in this blog post). There are soooooo many different things going on and a lot of stuff to remember. A good place to start could be certificate planning, which also means that you have to decide which IPs/hostnames you’ll be using in your own environment. Then again, I think the best place to start is planning network infrastructure/topology. First, consider whether you are going with a simple topology (Front-end, Edge server, Reverse proxy – we’re using this) OR a complex topology (multiple Front-ends, Edge servers, Reverse proxies). Second, It’s very important to have a working DMZ, and you should also know if you’ll be using (only) Public IP addresses or public IPs in combination with NATed ones. After you’ve got an answer to these questions it will be much easier planning for the other requirements. With this in mind, I’ll start off with the networking part. I’ll then move over to areas like DNS, certificates, actual Edge server installation, IIS ARR installation and finally some words about mobility and federation.
But first off, here’s a short explanation of what the Edge and Reverse proxy servers bring to the table:
Edge
Includes 4 modules:
-
Access Edge service. The Access Edge service provides a single, trusted connection point for both outbound and inbound Session Initiation Protocol (SIP) traffic.
-
Web Conferencing Edge service. The Web Conferencing Edge service enables external users to join meetings that are hosted on your internal Lync Server 2013 deployment.
-
A/V Edge service. The A/V Edge service makes audio, video, application sharing, and file transfer available to external users. Your users can add audio and video to meetings that include external participants, and they can communicate using audio and/or video directly with an external user in point-to-point sessions. The A/V Edge service also provides support for desktop sharing and file transfer.
-
XMPP Proxy service. The XMPP Proxy service accepts and sends extensible messaging and presence protocol (XMPP) messages to and from configured XMPP Federated partners.
Reverse proxy
The reverse proxy is required for the following:
-
To allow users to connect to meetings or dial-in conferences using simple URLs
-
To enable external users to download meeting content
-
To enable external users to expand distribution groups
-
To allow the user to obtain a user-based certificate for client certificate based authentication
-
To enable remote users to download files from the Address Book Server or to submit queries to the Address Book Web Query service
-
To enable remote users to obtain updates to client and device software
-
To enable mobile devices to automatically discover Front End Servers offering mobility services
-
To enable push notifications to mobile devices from the Office 365 or Apple push notification services
Source: https://technet.microsoft.com/en-us/library/gg425779%28v=ocs.15%29.aspx.
Networking / Network interfaces
I’m now assuming that you have:
-
A working Lync Server Standard/Enterprise 2013 (or Skype for Business) Front-end
-
A soon-to-become (Lync) Reverse Proxy server (Windows Server 2012 R2)
-
A soon-to-become Lync Edge server (Windows Server 2012 R2)
-
Talked to your network guys about the network infrastructure (IPs/DMZ). Hardware (F5) load balancers can be a whole different story for example.
-
Talked to your firewall guys about opening ports. I myself sat down with the Skype for Business Server 2015 Protocol Workloads printout and had a long discussion with a firewall guy. We/he got the job done without any hiccups. (It’s still working fine today 🙂 )
On the Reverse Proxy:
Assign one IP for the internal network adapter and one for the external network adapter. Internal and External should be in different subnets. One interface is communicating with the internet and the other one is communicating with your internal network/AD. Have a look at https://technet.microsoft.com/en-us/library/gg429707%28v=ocs.15%29.aspx or https://technet.microsoft.com/en-us/library/jj204691%28v=ocs.15%29.aspx for examples. (I’m not going into much DNS details (yet), but you could name these new IPs lyncproxy-int.domain.com and lyncproxy-ext.domain.com for example).
-
Set the default gateway on the external network adapter only
-
Assign static routes. From my experience, the information regarding this can be a bit difficult to understand. Let me copy/paste the information from the above link:
“Important: Similar to the Edge Servers, you set the default gateway on the external network adapter only. The default gateway will be the IP address of the router or external facing firewall that directs traffic to the Internet. For traffic that is destined from the reverse proxy to the internal facing network adaptor, you must use persistent static routes (such as the route command in Windows Server) for all subnets containing servers referenced by the web publishing rules. Setting a persistent route does not cause the computer to become a router. If IP forwarding is not enabled, the computer is acting only to direct specific traffic destined for another network to the appropriate interface. This is essentially setting two gateways – one as the default pointing to the external networks, and one for traffic destined to the internal interface and on to a router or other network.
However, creating persistent routes for all subnets may not be necessary if your network’s routers are configured to summarize routes. Create a persistent route to the network where the router is defined and use the router as the default gateway. If you are not sure how your network is configured and need guidance on what persistent routes need to be created, consult with your company’s Network Engineers.
The reverse proxy must be able to resolve the DNS host (A) records for the internal Director or Front End Server and next hop pool FQDNs used in the web publishing rules. As with the Edge Servers, for security reasons, we recommend that you do not configure a reverse proxy to use a DNS server located in the internal network. This means you either need DNS servers in the perimeter, or you need HOSTS file entries on the reverse proxy that resolves each of these FQDNs to the internal IP address of the servers”.
In plain English this means that you configure the external interface “normally”, as you would with any other external network interface in your infrastructure. You should define the gateway as the “IP address of the router or external facing firewall that directs traffic to the Internet”. Your network guys can help you with this if unsure (also see the next chapter). The internal side on the other hand should not have a default gateway – instead you configure static routes. I’ll try to explain this:
Example network subnets (defined by your network administrators):
External DMZ (16 addresses, all are not needed but room for expansion)
Range: 123.123.123.16/28
Mask: 255.255.255.240
GW: 123.123.123.30
Internal DMZ (16 addresses, all are not needed but room for expansion)
Range: 123.123.123.0/28
Mask: 255.255.255.240
GW: 123.123.123.14
Server configuration:
Example external network adapter configuration on the server:
IP: 123.123.123.20
Mask: 255.255.255.240
GW: 123.123.123.30
Example Internal network adapter configuration on the server:
IP: 123.123.123.3
Mask: 255.255.255.240
GW: no gw
We’re using split-brain DNS so the internal and external DNS names are the same. All IP’s are from a Class B chunk, and they’re all public IP’s that are defined as internal or external in the firewall/DNS. With the above configuration in place, you should now add a route to the internal interface on the reverse proxy server. This is done with the route add command (The –p switch make the changes persistent). Here’s an example using the above IP schema:
C:\>route add -p 123.123.17.0 mask 255.255.240.0 123.123.123.14
C:\>route add -p 123.123.90.0 mask 255.255.240.0 123.123.123.14
C:\>route add -p 123.123.154.0 mask 255.255.240.0 123.123.123.14
C:\>route add -p 123.123.207.0 mask 255.255.240.0 123.123.123.14
C:\>route add -p 123.123.106.0 mask 255.255.224.0 123.123.123.14
The above command example would make all of the above IP ranges take the route against the internal interface. All other IPs would take the external route. The above ranges are also defined as internal in DNS/firewall. Do the same for all of your internal IP ranges. This method is different when using NAT and/or non-split-brain configurations. (In case of NAT, your internal IPs are in the 192.168.x.x, 10.x.x.x, or 172.16.x.x. range). Perhaps a picture will tell more than words:
Fig 2. Internal and external overview. (Picture source: https://technet.microsoft.com/en-us/library/jj204756%28v=ocs.15%29.aspx)
This should be it for the networking part on the Reverse proxy server. Now we do the same on the Edge server.
On the Edge server:
The network configuration on the Edge server follow the same pattern as the Reverse proxy. I’m using three external IPs and one internal IP. This is by best practice design (https://technet.microsoft.com/en-us/library/gg412847%28v=ocs.15%29.aspx). If you are in a limited-budget-external-IP-dilemma, you can also make it work with one external IP (not including that option in this text however).
- Assign three external IPs
- one for SIP traffic
- one for AV traffic
- one for Web Conferencing
- Assign one internal IP
I’m not going much into DNS details now either, but you could name these new IPs lyncedge-int.domain.com, lyncedgesip-ext.domain.com, lyncedgeav-ext.domain.com andlyncedgewc-ext.domain.com for example. There’s nothing much to add here. Follow the same procedure as for the reverse proxy when configuring your internal and external network interfaces:
Server configuration:
Example Internal network adapter configuration on the server:
IP: 123.123.123.4
Mask: 255.255.255.240
GW: no gw
Example external network adapter configuration on the server:
IP1: 123.123.123.21
Mask: 255.255.255.240
GW: 123.123.123.30
IP2: 123.123.123.22
Mask: 255.255.255.240
GW: 123.123.123.30
IP3: 123.123.123.23
Mask: 255.255.255.240
GW: 123.123.123.30
Now add the same routes as you did on the reverse proxy. There you have it, we can now move over to the DNS part.
DNS
I assume that you by now have figured out your topology and configured networking on the involved servers. Good, that’s one step in the right direction. You might have noticed that I haven’t talked much about host names, only IP addresses. This is mostly because you can configure the networking part this far without knowing (almost) any host names. (Of course you most certainly will ask for a hostname at the same time you get an IP address, but anyways).
I have to say that DNS was one of the most confusing/difficult/challenging/painful parts in this whole configuration/deployment. There were tons and tons of misleading/wrong information, and it required countless hours of testing. Anyways, I’ll spare you the DNS-pain and tell you about our configuration in a while. But before I do, I make you read some homework. Here are a couple of interesting links (with or without errors):
https://technet.microsoft.com/en-us/library/dn951397.aspx
https://technet.microsoft.com/en-us/library/gg398758%28v=ocs.15%29.aspx
http://jackstromberg.com/2013/06/lync-2013-dns-settings/
https://technet.microsoft.com/en-us/library/gg425874%28v=ocs.15%29.aspx
https://technet.microsoft.com/en-us/library/gg398287%28v=ocs.15%29.aspx
http://msunified.net/2013/08/07/lync-client-sign-in-and-dns-records-recommendations/
Let me start off by saying that I like the jackstromberg article. All my testing was actually based on the DNS table from that article. However those records were also a bit confusing, and some even unnecessary. Here are our DNS records with comments:
Internal DNS:
Fig 3. Internal DNS
No other records are required for our specific configuration/environment (at the moment). SRV records are a thing of the past and only needed when working with Lync 2010 clients. See http://blog.schertz.name/2012/12/lync-2013-client-autodiscover/ for more information. If you are going to use federation however (which we probably are in the future), you SHOULD set up SRV records (though not needed if manually entering servers). See: http://windowsitpro.com/skype-business/understanding-main-types-federation-skype-and-lync-environments. As you can see, Allowed Partner Server (Direct Federation) works without SRV records but specifying the records when you federate will probably still make your life easier.
I will now also make a statement about the lyncdiscover.mydomain.com record in the Internal DNS. You can read on many, MANY places on the Internet that you should have this record present in the internal DNS so that mobility works. I can confirm that our users mobile phones (WP, iOS, Android) work just FINE without this record. The key is to have the external Web Services record present in the internal DNS (webext.mydomain.com) and point it to the reverse proxy. If you DO use lyncdiscover in the internal DNS, ALL traffic will go through the proxy. This is probably not a desirable configuration. Yes, I’ve seen this “live” in our environment so I know what I’m talking about. The “problem” went away after we removed the lyncdiscover.mydomain.com record from the internal DNS. Good info about this:
http://blog.schertz.name/2013/07/understanding-lync-2013-mobility/
http://www.ucprimer.com/tech-blog/5-essential-tips-for-deploying-lync2013-mobility-service
https://technet.microsoft.com/en-us/library/hh690030%28v=ocs.15%29.aspx
Read the above links CAREFULLY and you’ll have a MUCH better understanding, believe me 🙂Again, this setup works FOR US. I’m not saying that the lyncdiscover.mydomain.com record should be removed from every internal DNS configuration out there.
External DNS:
Fig 4. External DNS.
External DNS was much more straight forward. Comments are included in the picture.
The DNS records are (as you can see) a bit different for the external network/outside world compared to the internal network. All external traffic goes through the reverse proxy, which in turn use URL rewrites to connect to the corresponding URLs on the inside network. (I’ll leave the URL rewrite / ISS/ARR discussion for a later chapter).
Hosts file:
You have probably noticed that the webext.mydomain.com record is present in BOTH the internal and external DNS. The reason for this is mobile devices. Mobile devices need to access the mobility service, and they do that ONLY from the outside. I’m yet again referring to the Mobility service flow using AutoDiscover (picture) at https://technet.microsoft.com/en-us/library/hh690030%28v=ocs.15%29.aspx. Lyncs own autodiscover feature will know if the client is on the internal or external network based on the lyncdiscover/lyncdiscoverinternal record. However, it’s a whole different story with webext.mydomain.com. If this URL is accessed from either the inside or outside network, the client is unable to know it’s final destination. This is because you’re pointing the client (in both cases) to the reverse proxy, which in turn point to the same URL internally and externally. This means that you’ll end up in an endless loop. To solve this you’ll edit the hosts-file.
You also have to add a local DNS record for lyncdiscover, otherwise this record will remain unresolvable as it’s not present in the internal DNS. This was all a big mystery for me, as the documentation seldom mentioned this dilemma. I got an idea after hours of googling though – the holy hosts-file. Thanks to http://kressmark.blogspot.fi/2015/06/windows-2012-r2-iis-85-and-arr-30-as.html for the idea. This was by no means a big surprise, but you’ll get lost (in DNS) after hours and hours of testing. Believe me.
This means that you’ll have to add local DNS records on the reverse proxy. Fire up notepad and edit the C:\Windows\System32\drivers\etc\hosts –file on the reverse proxy. Add the following:
123.123.123.10 webext.mydomain.com, where 123.123.123.10 is the IP of your Front-end server.
123.123.123.10 lyncdiscover.mydomain.com, where 123.123.123.10 is the IP of your Front-end server.
(Meet and dialin are already resolvable by internal DNS and correctly points to the Front-end).
Now when a client resolves webext (internally or externally), it always gets sent to the reverse proxy. The reverse proxy in turn resolves webext to the front-end via the hosts-file. Lyncdiscover in turn won’t be resolvable internally after it reaches the reverse proxy if no hosts file-record is added. There you have it – all your DNS problems solved 🙂
Certificates
Certificates and DNS go somewhat hand-in-hand as you need to know which hostnames you’ll be using in the certificates. I’d probably start off by reading again. Some information to get you started:
https://technet.microsoft.com/en-us/library/jj204747%28v=ocs.15%29.aspx
https://technet.microsoft.com/en-us/library/jj205381%28v=ocs.15%29.aspx
http://skype4b.uk/2015/08/29/skype-for-business-certificate-requirements-the-definitive-guide/
http://social.technet.microsoft.com/wiki/contents/articles/24210.demystify-lync-2013-server-internal-certificate-requirements.aspx
Take your time to read and plan – this way you’ll be rewarded in the end. The above links discuss both internal and external certificates. They also discuss the differences between the reverse proxy and the Lync Edge certificates.
We’re not completely going by best practice regarding the certificates. We’re using external certificates on the internal front-end server. This is due to the fact that we already had an external certificate installed on the Front-end. It doesn’t do much harm either, and at least for us it’s not an extra expense.
Without further stories I’ll present our “certificate solution”. We’re using one certificate per interface on the edge server, but you could also use just one certificate will all hosts included. This would be more expensive due to the fact that you’ll have to pay extra for additional SAN-names. (You DON’T need a certificate for the AV-interface on the Edge server).
Fig 5. Edge and Reverse proxy certificate chart.
To add to this list, our Front-end needed to get its public certificate renewed with the added host webext.domain.com. Before the renewal, it included public certs for lyncdiscover, lyncdiscoverinternal, meet, dialin and sip. Now it includes all those + webext. Webext is needed for the external Web Services (externally accessing the front-end). More about that later on.
The following chapters has information about how to install the certificates on the servers.
Configuring the Lync Front-end Server for Edge / Installing the Edge Server / Edge Server Certificate Installation
Congratulations If you’ve had the energy to read this far. It’s now finally time to install the Edge server 🙂 Much of the Edge installation/configuration is actually tied to the Front-end server however. You’ll start by making changes to your current topology and then export/publish the topology on the Edge server. Like you’ve probably noticed before, it’s the prep work that takes most of the time. (Internet is full of articles on how to install an Edge server). That said, I happen to like the post at http://www.sherweb.com/blog/installing-lync-2013-edge-server/ and our installation is based on this article. However, there are some differences. In Step 6, the article tells you to request internal and external certificates from the setup itself. We didn’t do it this way because our internal CA isn’t an “online certification authority” (it doesn’t respond to online web requests due to security reasons). Instead we made an offline request and signed it manually on the CA. A little bit more hassle, but worked just fine in the end.
We didn’t use the certificate request wizard for the external certificate either, as those gets created by our “certificate guy”. He uses his own methods and just delivers a fully working certificate. I won’t go into the details, but it works. So in the end, whatever floats your boat and can get you the correct certificates is fine 🙂
If using this “manual method” (alternative to Step 6 in the guide), you must manually install the certificate(s) in the certificate store(s) before continuing. This is by no means difficult. For the internal/external certificates do the following:
- Fire up “mmc.exe” on the Edge server
- Add the certificates Snap-in. Select computer account –> local computer and click OK.
- Right click on the Personal –> Certificates folder.
- Select All Tasks… Import
- Imported certificate will show up in this location.
- Move the different certificates to their corresponding places (Personal, Trusted Root Certification Authorities, Intermediate Certification Authorities). See screenshots/figures below.
Fig 6. Personal certificate (computer)
Fig 7. Trusted Root CA certificate
Fig 8. Intermediate CA certificate
- Done. External certificates are also showing in this list as I have imported them in the same manner.
Now, back to the installation post at http://www.sherweb.com/blog/installing-lync-2013-edge-server/.
Some of my own notes:
- Followed the guide.
- Network specifications were OK.
- Software specifications were OK. Didn’t (need to) install Windows Identity Foundation on the Front-end. (It should be installed as a pre-requirement on the Edge server however).
- Fired up Topology Builder on the front-end and followed the guide.
- FQDN of the new edge pool should match the FQDN (CN) on the internal certificate: lyncedge-int.domain.com from certificate chart
- Access Edge service should match the FQDN (CN) on the external certificate: lyncedgesip-ext.domain.com
- Web Conferencing Edge service should match the FQDN (CN) on the external certificate: lyncedgewc-ext.domain.com
- A/V Edge service should match the FQDN (CN) on the external certificate: lyncedgeav-ext.domain.com
- NOTE: This is a “DNS/certificate-thing”. Whatever certificate CN-record you created for the above services should be used. You can swap CN and SAN records for a more “clean” name, i.e. lyncedgeav-ext.domain.com vs. av.domain.com. (See above DNS/certificate chart and the note about switching places between CN and SAN).
- Enabled federation (not xmpp yet though)
- Defined internal and external IP addresses
- Defined Next hop pool: our front-end
- Changed the External web services in the Topology builder to match the one we have defined in DNS and in the certificate (would be webext.domain.com in the example).
Fig 9. External web services
-
- Published the topology
- Exported the configuration (step 4 in guide).
- Moved over to the Edge server itself
- Installed Lync server and imported the configuration (step 5 in guide)
- The certificates were already installed in the certificate store so no need to request certificates (step 6)
- Defined the existing certificates
- Started services
- Done! (no need for step 8 and 9, yet)
Installing and configuring the Reverse Proxy server / Reverse Proxy server Certificate Installation
I’m now assuming that you have a working Edge server. You can install the reverse proxy server without a working Edge server also, but installing the Edge server first makes it easier to test the reverse proxy functionality right after the installation. First some homework/reading:
https://technet.microsoft.com/en-us/library/jj204691%28v=ocs.15%29.aspx
https://technet.microsoft.com/en-us/library/gg398069%28v=ocs.15%29.aspx
https://blogs.technet.microsoft.com/nexthop/2013/02/19/using-iis-arr-as-a-reverse-proxy-for-lync-server-2013/
http://jackstromberg.com/2014/11/tutorial-deploying-a-reverse-proxy-for-lync-server-2013/
http://kressmark.blogspot.fi/2015/06/windows-2012-r2-iis-85-and-arr-30-as.html
https://thamaraw.com/2013/07/15/publish-lync-server-external-web-services-using-iss-arr/
I decided to go with a combination of the Microsoft guide and the jackstromberg one this time. In the end, it worked perfectly. I had lots of problems and headaches down the line, but this time it had nothing to do with the guides (rather it had to do with typos and a non-working ARR that had to be reinstalled).
Before following the (Microsoft) guide however, we have to install the certificates the same way we did on the Edge server. The reverse proxy certificates are of course different, but I’m assuming that you have requested them at the same time as the Edge certificates. Just follow the Edge steps (fire up mmc.exe and so on) and you’re good to go. One different step is that you have to bind the certificates in IIS, otherwise they won’t be used when clients connect via the reverse proxy. It’s rather easy, let me show you some screenshots:
Fig 10. Adding https bindings in IIS.
Fig 11. Adding https bindings in IIS. Remember to add both the internal and the external interface, with their own certificate.
Now continue following the guides, or use another guide of your choice.
I used the following simple URLs:
- meet.domain.com
- dialin.domain.com
- webext.domain.com
- lyncdiscover.domain.com
All good, tested and working! 🙂
A note from my own experiences: ARR is VERY STRICT regarding the URL rewrite rules. If something isn’t working, be sure to double check the rules!
Mobility
I wasn’t quite sure if there was a need for a separate mobility chapter as I’ve covered this area quite well in the DNS and certificate chapters. I guess a couple of lines won’t do no harm however. I’ll once again start by giving you a nice list of homework/reading:
https://technet.microsoft.com/en-us/library/hh690030%28v=ocs.15%29.aspx
http://blog.schertz.name/2013/07/understanding-lync-2013-mobility/
http://www.ucprimer.com/tech-blog/5-essential-tips-for-deploying-lync2013-mobility-service
http://www.lyncexch.co.uk/lyncdiscover-and-auto-discovery-deeper-dive/
http://blog.schertz.name/2012/12/lync-2013-client-autodiscover/
http://www.ucprimer.com/tech-blog/5-essential-tips-for-deploying-lync2013-mobility-service
https://blogs.technet.microsoft.com/nexthop/2012/02/21/troubleshooting-external-lync-mobility-connectivity-issues-step-by-step/
https://ucgeek.co/2014/02/lync-mobility-dns-records/
http://www.lyncinsider.com/lync-mobile/external-lync-2013-users-need-a-2013-edge-server-to-use-mobility-services/
https://social.technet.microsoft.com/Forums/lync/en-US/daacb3f7-4c09-4d2d-aca4-354cba7a6acc/mobility-without-using-edge-server-2013?forum=lyncdeploy
https://social.technet.microsoft.com/Forums/lync/en-US/b6519fb1-ce56-4977-aaec-3ad23900d8e6/lync-2013-topology-change-now-require-a-reverse-proxy?forum=lyncdeploy
http://windowsitpro.com/lync/decoding-lyncs-endpointconfigurationcache-file (deleting cache)
Much of the mobility stuff has to do with the fact that the mobility service isn’t working properly via lyncdiscoverinternal. Instead you configure the mobile devices to go the external way, via the reverse proxy. See the following picture:
Fig 12. Lync mobility (source: http://www.ucprimer.com/tech-blog/5-essential-tips-for-deploying-lync2013-mobility-service)
The mobility bit was a big headache, but in the end we got it working in a desirable way. The secret was to remove lyncdiscover.domain.com from the internal DNS (against many recommendations). See the DNS chapter for more information.
Federation
We’re definitely interested in federation, but we haven’t federated with any partners yet. It’s no harm reading about federation though, and in the end it will be much easier setting it up once you’ve done your homework. I’ve done my homework, so why wouldn’t you 🙂 Here you go:
http://windowsitpro.com/skype-business/understanding-main-types-federation-skype-and-lync-environments
https://technet.microsoft.com/en-us/library/jj204800%28v=ocs.15%29.aspx
https://blogs.technet.microsoft.com/uclobby/2013/09/30/difference-between-open-and-closed-federation-in-lync-20102013/
https://technet.microsoft.com/en-us/library/jj205134.aspx
The Edge server is already enabled for federation, but the front-end is not. This is easily fixed in the Topology builder once we/you decide to federate:
Fig 13. Enable federation on the Front-end.
In addition to this, it’s also recommended that you add a DNS SRV record (_sipfederationtls._tcp.domain.com).
And finally here’s a screenshot I took sometime in the middle of the whole deployment. As I’ve stated before, there were lots and lots of googling and homework to be done 🙂
Fig 14. Google is your friend. Don’t believe everything you read though…
That’s Firefox with Tab Mix Plus and Multirow Bookmarks Toolbar Plus Extensions, btw.
This quite much summarizes the Lync Edge and Reverse Proxy server deployment. Hope you’ve enjoyed reading 🙂