Exchange 2016 On The Horizon

With Exchange 2016 on the horizon (RTM in Q4 2015) I’ve been prepping the lab infrastructure and dusting of some parts of the Exchange 2010/2013 lab deployments to make sure I’m ready to start testing an migration to Exchange 2016. While Office 365 offers great value for money sometimes there is no option to switch over completely and a (used) hybrid scenario is the way to go.  This can be regulations, politics, laws, etc. No matter what we have to come up with a solutions that satisfy all needs as well as possible. Even in 2015 or 2016 this will mean on premises e-mail. This is no my “default” option when it comes to e-mail in anno 2015, but it’s still a valid option and choice. So they can get the best of both worlds and be compliant. Is this the least complex solution? No, but it gives them the compliancy they need and/or want. It’s not my job to push companies 100% to the cloud. That’s for the CIO to decide and for cloud vendors to market/sell. I’m in the business of helping create the best possible solution to the challenge at hand.

Figure: Exchange 2016 Architecture © Microsoft

The labs were setup to test & prepare for production deployments. It all runs on Hyper-V and it has survived upgrades of hypervisor (half of the VMs are even running on Windows Server 2016 hosts) and the conversion of the VHDX to VHDX.  These labs have been kept around for testing and trouble shooting. There are fully up to date. It’s fun to see the old 2009 test mails still sitting in some mail boxes.

image

Both Windows NLB and Kemp Technologies Loadmasters are used. Going forward we’ll certainly keep using the hardware load balancing solution. Oh, when it comes to load balancing, there only the best possible solution for your needs in your environment. That will determine which of the various options you have you’ll use. In Exchange 2016 that’s a will be very different from Exchange 2010 in regards to session affinity, affinity is no longer needed since Exchange 2013.image

In case you’re wondering what LoadMaster you need take a look at their sizing guides:

Another major change will be the networking. On Windows Server 2012 R2 we’ll go with a teamed 10Gbps NIC for all workloads simplifying the setup.  Storage wise one change will be the use of ReFS, especially if we can do this on Storage Spaces. The data protection you get from that combination is just awesome. Disk wise the IOPS have dropped yet even a little more so that should be OK. Now, being a geek I’m still tempted to leverage cheap / larger SSDs to give flying performance Smile. If possible at all I’d like to make it a self contained solution, so no external storage form any type of SAN / centralized storage. Just local disks. I’m eyeing the DELL R730DX for that purpose. Ample of storage, the ability to have 2 controllers and my experience with these has been outstanding.

So no virtualization? Sure where and when it makes sense and it fits in with the needs, wants and requirements of the business.  You can virtualize Exchange and it is supported. It goes without saying (serious bragging alert here) that I can make Hyper-V scale and perform like a boss.

Exchange 2010 SP3 Rollup 5 Added Support for Windows Server 2012 R2 Active Directory

6 weeks ago (February 25th 2014) Microsoft finally took away the last barrier to upgrading some of our Windows Server 2012 Active Directory Environments to R2.  Most of them are still running Exchange 2010 SP3 and not Exchange 2013. The reason is that Exchange 2013 was not deployed is whole other discussion Eye rolling smile.

However that dis mean that until the release of  Exchange Server 2010 SP3 Update Rollup 5 last month we could not upgrade Active Directory to Windows Server 2012 R2. Rollup 5 brought us support for exactly that. We can now:

  • Support Domain Controllers running Windows Server 2012 R2
  • Raise the Active Directory Forest Function Level and Domain Functional Level to Windows Server 2012 R2

Please note that you cannot deploy Exchange Server 2010 (SP3 RU5) on Windows Server 2012 R2 and you’ll probably never will be able to do that. I’m not sure Microsoft has any plans for this.

Now our office moves have been concluded, meaning I can get back to IT Infrastructure instead of being an glorified logistics & facility peon, we’re doing the upgrade.

This also means we can move the Active Directory environments to the latest version so we have the best possible position for any future IT projects at very low risk. The environments are already at W2K12 functional level. If the budgets get so tight they lose/scrap EA or volume licensing it also allows them to run at this level for many years to come without causing any blocking issues.

Upgrading Exchange 2010 SP1 To SP2

Here is a step by step walk trough of an Exchange 2010 SP2 installation. I needed to document the process for a partner anyway so I might as well throw on here as well. Perhaps it will help out some people. The Exchange Team announced Exchange 2010 SP2 RTM on their blog recently. There you can find some more information and links to the downloads, release notes etc. You will also note that the Exchange 2012 TechNet documentation has SP2 relevant information added. if you just want to grab the bits get them here; Microsoft Exchange Server 2010 Service Pack 2 directly from Microsoft
 
Exchange 2010 SP1 and SP2 can coexist for the time you need to upgrade the entire organizations. Once you started to upgrade it’s best to upgrade all nodes in the Exchange Organization as fast as you can to SP2. That way you’ll have all of them on the same install base which is easier to support and trouble shoot. Before I did this upgrade in production environments I tested this two time in a lab/test environment. I also made sure  anti virus, backup and other agents dis not have any issues with Exchange 2010 SP2. Nothing is more annoying then telling a customer his Exchange Organization has been upgraded to the lasted and greatest version only to follow up on that statement with the fact the backups don’t run anymore.

You can install Exchange SP2 easily via the setup wizard that will guide you through the entire process. There are some well documented “issues” you might see but these are just about the fact you need IIS 6 WMI compatibility for the CAS role now and the fact that you need to upgrade the Active Directory Schema. Please look at Jetze Mellema’s blog for some detailed info & at Dave Stork’s blog post for consolidated information on this service pack.

Changing the Active Directory schema is a big deal in some environments and you might not be able to do this just as part of the Exchange upgrade. Perhaps you need to hand this of to a different team and they’ll do that for you using the  command “setup /prepareschema” as shown below.

image

You’ll have to wait for them to give you the go ahead when everything is replicated and all is still working fine. Below we’ll show you how you can do it with the setup wizard.

Order of upgrade is a it has been for a while

  1. CAS servers
  2. HUB Transport servers
  3. If you run Unified Messaging servers upgrade these now, before doing the mailbox servers
  4. Mailbox servers
  5. If you’re using Edge Transport servers you can upgrade them whenever you want.

Let’s walk through the process with some additional information

Once you’ve download the bits and have the Exchange2010-SP2-x64.exe file click it to extract the contents. Find the  setup.exe and it will copy the files it needs to start the installation.

image

 

You then arrive at the welcome screen where you choose “Install Microsoft Exchange Server Upgrade”

image

 

The setup then initializes

image

 

You get to the Upgrade Introduction screen where you can read what Exchange is and does Smile. I hope you already know this at this stage. Click Next.

image

 

You accept the EULA

image

 

And watch the wizard run the readiness checks

image

 

In the lab we have our CAS/HUB servers on the same nodes, so the prerequisites are checked for both. The CAS servers in Exchange 2010 SP2 need the IIS 6 WMI Compatibility Feature. If you had done the upgrade from the CLI you would have to run SETUP /m:upgrade /InstallWindowsComponents and you would not have seen this error as it would have been taken care of installing the missing components. When using the GUI you’ll see the error below.

image

You can take care of that by installing this via “Add Role Services” in Server Manager for the Web Server (IIS) role.

image

image

Or you can use our beloved PowerShell with the following commands:

  • Import-Module ServerManager
  • Add-WindowsFeature Web-WMI.

image

Now we have the IIS 6 WMI compatibility issue out of the way we can rerun the readiness checks and we’ll get all green check marks.

image

So we can click on “Upgrade” and get the show on the road. The first thing you’ll see this step do is “Organization Preparation”. This is the schema upgrade that is needed for Exchange 2010. If you had run this one manually it would not have to this step and you’ll see later it only does is for the first server you upgrade (note that it is missing form the second screen print, which was taken from the second CAS/HUB role server). I like to do them manually and make sure Active Directory replication has occurred to all domain controllers involved. If I use the GUI setup I give it some time to replicate.

Intermezzo: How to check the schema version

You can verify after having run SP2 on the first node or having updated the schema manually that this is indeed effective by looking at the properties of both the domain and the schema via ADSIEdit or dsquery.

The value for objectVersion in the properties of “CN=Microsoft Exchange System Objects” should be 13040. This is the domain schema version. Via dsquery this is done as follows: dsquery * “CN=Microsoft Exchange System Objects,DC=datawisetech,DC=corp” -scope
base -attr objectVersion

image

The rangeUpper property of “CN=ms-Exch-Schema-Version-Pt,cn=schema,cn=configuration,<Forest DN>” should be 14732. You can also check this using dsquery * CN=ms-Exch-Schema-Version-Pt,cn=schema,cn=configuration,<Forest DN> -scope base –attr rangeUpper tocheck this value

image

Note that you might need to wait for Active Directory replication if you’re not looking at the domain controller where the update was run. If you want to verify all your domain controllers immediately you can always force replication.

Step By Step Continued

First CAS/HUB roles server (If you didn’t upgrade the schema manually)

image

Additional CAS/HUB roles server

image

… it takes a while …

image

But then it completes and you can click “Finish”

image

We’re done here so we click “Close”

image

When you run the setup on the other server roles like Unified Messaging, Mailbox and Edge the process is very similar and is only different in the fact it checks the relevant prerequisites and upgrades the relevant roles. An example of this is below for a the mailbox role server.

image

image

In DAG please upgrade all nodes a.s.a.p. and do so by evacuating the databases to the other nodes as to avoid service interruption. The process to upgrade DAG member is described here: http://technet.microsoft.com/en-us/library/bb629560.aspx

  • Upgrade only passive servers Before applying the service pack to a DAG member, move all active mailbox database copies off the server to be upgraded and configure the server to be blocked from activation. If the server to be upgraded currently holds the primary Active Manager role, move the role to another DAG member prior to performing the upgrade. You can determine which DAG member holds the primary Active Manager role by running Get-DatabaseAvailabilityGroup <DAGName> -Status | Format-List PrimaryActiveManager.
  • Place server in maintenance mode Before applying the service pack to any DAG member, you may want to adjust monitoring applications that are in use so that the server doesn’t generate alerts or alarms during the upgrade. For example, if you’re using Microsoft System Center Operations Manager 2007 to monitor your DAG members, you should put the DAG member to be upgraded in maintenance mode prior to performing the upgrade. If you’re not using System Center Operations Manager 2007, you can use StartDagServerMaintenance.ps1 to put the DAG member in maintenance mode. After the upgrade is complete, you can use StopDagServerMaintenance.ps1 to take the server out of maintenance mode.
  • Stop any processes that might interfere with the upgrade Stop any scheduled tasks or other processes running on the DAG member or within that DAG that could adversely affect the DAG member being upgraded or the upgrade process.
  • Verify the DAG is healthy Before applying the service pack to any DAG member, we recommend that you verify the health of the DAG and its mailbox database copies. A healthy DAG will pass MAPI connectivity tests to all active databases in the DAG, will have mailbox database copies with a copy queue length and replay queue length that’s very low, if not 0, as well as a copy status and content index state of Healthy.
  • Be aware of other implications of the upgrade A DAG member running an older version of Exchange 2010 can move its active databases to a DAG member running a newer version of Exchange 2010, but not the reverse. After a DAG member has been upgraded to a newer Exchange 2010 service pack, its active database copies can’t be moved to another DAG member running the RTM version or an older service pack.

Microsoft provides two PowerShell scripts to automat this for you. These scripts are StartDagServerMaintenance.ps1 and StopDagServerMaintenance.ps1 to be found in the C:Program FilesMicrosoftExchange ServerV14Scripts folder. Usage is straight forware just open EMS, navigate to the scripts folder and run these scripts for each DAG member like below.

  1. .StartDagServerMaintenance –ServerName “Invincible”
  2. Close the EMS other wise PowerShell will hold a lock files that need to be upgraded (same reason the EMC should be closed) and than upgrade of the node in question
  3. .StopDagServerMaintenance –ServerName “invincible”

image

Voila, there you have it. Happy upgrading. Do you preparations well and all will go smooth.

A Brighter Future For Public Folders?

The Exchange Team posted a blog entry asking for feedback on how we use public folders. Nice to see they are taking an interest again. The past 4 years the mantra was “move away from them”, “do it now while you still have the time”, etc. SharePoint was always put forwards as number one replacement option. For some scenarios this is indeed a good choice but let’s face it, for some public folder uses there is no decent replacement and that hurts us as they haven’t seen any decent improvements in the last 2 Exchange releases. I know public folders have always been a bit problematic and finicky for us administrators. They tend to need a bit of voodoo and patience to trouble shoot and get running smoothly (see  blog post by me for an example of this). But instead of using that of an excuse to get rid of them they could also choose to invest in making them as reliable and robust as mail databases. Giving them the same high availability features might also be a welcome improvement, especially now with DAGs in Exchange 2010.

Especially in the Exchange 2007 era Microsoft was promoting getting rid of them actively. But they are still around because so many people use them and they have not decent alternative for all scenarios. In that respect they do listen to their customers. But we want improvements. Some of the functionality we need is there but we really need more robust, reliable and high available public folders. As as shared mail instrument for both sending and receiving mail in a team public folders beat shared mailboxes and SharePoint any time.  It also shines for maintaining a shared repository of contacts. I’m not a proponent of using public folder for a document repository but I understand that its relative simple usage and data protection via replicas still sounds attractive to some versus the complexity of SharePoint. Sure SharePoint has more to offer but perhaps they don’t need those capabilities and to make matters even less attractive; it’s quite an effort to migrate from public folders to SharePoint.

So that left us public folders users feeling a bit abandoned with a message of get out but no easy path to go anywhere else that serves all our needs. So until today all my customers are still and want to  keep using public folders. They are a worried however that one day they will be left out in the cold. But perhaps there is a better future on the horizon for public folders.  They are asking us to “Help us learn more about how you use public folders today!” in that blog post. The emphasis is on “usage scenarios, folder management habits or thought process around public folder data organization”. So if you need and use public folders in any way and you’d like for them to get more attention and evolve into more robust and functional instruments give Microsoft your feedback. Exchange 2010 has brought us great features & very affordable high availability together with support for virtualization. Now we either need a better alternative to public folders than the ones we got now or (my preference) we need better public folders. Since consumption of public folders occurs mostly in Outlook I would suggest the latter. And while we’re asking, bring back access to folder shares in OWA Winking smile.