Kick start ADFS when your self- signed certificates have expired already

I recently had to do some lab work on a Windows Server 2012 R2 ADFS farm to prep for a migration to Windows Server 2016.  Due to some storage shortage and some upgrades and migrations (all hardware in the lab runs Windows Server 2016) I had parked my Windows Server 2012 R2 ADFS farm offline.

So when I copied them back to my cluster and imported them I knew I had to make sure the domain was OK. This is easy enough, just run:

Reset-ComputerMachinePassword [-Credential Mydomain\bigadmin -Server MyDC01

That worked like a charm and soon enough my 2 VMs where up an running happily in the domain. I did have some issues however. My AFDS servers had been of line long enough before the expiration of the token-decrypting and the token-signing certificates to not yet have generate the new certificates for auto renewal and long enough to have them expire already. Darn!

image

The result was a bunch of errors in the event log as you might expect and appreciate.

image

An error occurred during an attempt to build the certificate chain for configuration certificate identified by thumbprint ’26AFDC4A226D2605955BF6F844F0866C14B1E82B’. Possible causes are that the certificate has been revoked or certificate is not within its validity period.
The following errors occurred while building the certificate chain: 
MSIS2013: A required certificate is not within its validity period when verifying against the current system clock.

But this also raised the question on how to get the ADFS servers back in a working condition. Normally these are generated automatically close to the expiration date of your existing certs (or at the critical threshold you configured). So I disabled / re-enabled auto certificate rollover but does actually does it even kick in if you have already expired? That I don’t know and I really had no time to wait hours or days to see what happens.

Luckily there is a command you can issue to renew the certificates immediately. This is the same command you can use when you have disabled auto rollover and need it re-enabled. That works normally after some patience.

Update-AdfsCertificate -Urgent

The result was immediate, the self signing certs were renewed.

image

And we can see this in the various entries in the event log

image

Do note that this command will cause a disruption of the service with your partners until they have refreshed the information from your federation metadata – or in the case this isn’t or can’t be leveraged, manually updated. In my case I had a “service down” situation anyway, but in normal conditions you’d plan this and follow the normal procedure you have in place with any partner that need your ADFS Services.

Easily migrating non-AD integrated DNS servers while preserving server names and IP addresses

Introduction

I’ll show you the quickest way to move an existing public advertising DNS deployment on Windows Server 2012R2, generation 1 virtual machines (1 primary DNS server and 1 or more secondary DNS Servers) to Windows 2016 RTM generation 2 VMs. On top of this we will preserve the sever names and the IP addresses. This makes the migration easier and it doesn’t burden anyone with updating IP addresses or FQDN of services pointing to the existing public advertising DNS service. Basically the result is the best possible for everyone involved.

Step by Step

We start by preparing a sysprepped VHDX of Windows 2016 with all the updates installed and any tools that are sysprep compatible and that you want or need on your VMs. This will allow us to make the move fast. As we want our new DNS VMs to be generation 2 VMS, make sure you use a generation 2 VM to create the syprepped OS VHDX.

The process we describe below is the same for each of the involved DNS servers. You start with the secondary VMs and end with the primary VM. This is just a form risk reduction, it’s smart to start with the secondary as it’s less critical than the primary where you make the changes.

Log on to the old, source VM and do the following

  1. Create a Folder to store the migration data and Info, i.e. C:\DNSMigrateServer01
  2. Open an elevated command prompt
  3. Run Ipconfig /all > C:\DNSMigrateServer01\Server01TCPIPinfo.txt this gives you the IP info you need for future reference.
  4. Run reg export HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\DNS\Parameters C:\DNSMigrateServer01\Dns-Service.REG
  5. Run reg export HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\DNS Server” C:\DNSMigrateServer01\Dns-Software.REG
  6. In some cases, rarely for most deployments, you’ll need to also copy all files under each custom database directory on the old DNS server by manually reading from the registry at the following path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DNS\Parameters\DatabaseDirectory If you have these also copy the directory to C:\DNSMigrateServer01. Normally when you have custom DNS database locations this is not by accident and should be well documented.
  7. Run xcopy %windir%\system32\dns C:\DNSMigrateServer01 /s This copies the content of your DNS folder (normally C:\Windows\System32\dns) to your migration folder. Note that you don’t need to copy the samples sub folder. Even the backup folder is not really needed. Just create a new backup when needed on the news DNS servers.
  8. Copy the C:\DNSMigrateServer01 from your old DNS Server to your desktop or some file share for safe keeping. You’ll need to copy this into the new DNS Server later. Note it contains your IP information, your registry exports and your DNS files.

You now have everything you need form the old DNS Server. So now we’ll decommission it, but before we do so we’ll make sure we have the options to recover it if needed.

  1. Make sure you have a backup or have made on recently (you do trust your ability to restore, right?)
  2. Shut down the VM and for good measure and fast recovery you might want to export the VM for quick import.
  3. Remove the VM from Failover Clustering if it’s clustered.
  4. Now remove the VM from Hyper-V Manager. Note this doesn’t delete the virtual disk files.
  5. Remove the old VHDX (you have an export and a backup) and replace it with your sysprepped W2K16RTM VHDX that has all the updates already. Rename that VHDX to something sensible like server01disk01.vhdx.
  6. Create a new generation 2 VM with the same name as the old one, select the required memory settings, choose to use an existing VHDX and point it to your sysprepped VHDX.
  7. Start the VM
  8. Go through the mini wizard and log in to it.
  9. Configure the NIC with the same setting as your old DNS Server
  10. Rename the VM to the old DNS VM name and join the domain.
  11. Restart the VM
  12. Login to the new DNS VM
  13. Install DNS
  14. Copy the C:\DNSMigrateServer01 you saved from your old DNS Server into the new one
  15. Open an elevated command prompt and run
    • Stop the DNS Server service by running net stop “DNS Server”
    • Double click the Dns-Service.REG and merge them into the registry

clip_image001

    • Double click the Dns-Software.REG files and merge them into the registry.

clip_image002

    • Copy all the files under C:\DNSMigrateServer01 to %windir%\System32\DNS
    • Start the DNS Server service by running net start “DNS Server”

Congratulations, you now have a new generation 2 VM running DNS on Windows Server 2016 with the same name and IP configuration as the old one. You now want to validate it’s working. To do so on the primary DNS server update the serial number in the start of authority (SOA) tab of the zone properties. I normally use YearMonthDayXX.

clip_image003

This will allow you to check whether the zone transfers to your migrated DNS server work. Normally all is just fine. In case things went horribly wrong you can import the VMs you exported or restore the backups. If your VMs are domain members and as you have reused the VM name, you’ll need to reestablish its domain member ship but that’s easily done.

Now repeat the above process for all the reaming secondary DNS Server and finally for the primary DNS server. Until you’ve done them all.

Conclusion

You do this process for every DNS Server and finally for your primary DNS server. That’s it. You’re in business and you have achieved 2 goals. You’re DNS VMs have been move to generation 2 and are running on a clean install of Windows Server 2016. All this without having to reconfigure DNS zone and transfers and while maintaining your DNS server names and IP addresses. Life is good.

Post MVP Summit – Back to reality

Coming home after the MVP Global Summit is a moment of reflection, or better the trip home is. The Summit is a time of intense interaction with peers who are very varied bunch of experienced technologists. Next to their hands on Microsoft stack expertise they also bring their experience with other technologies and companies. This gives us the opportunity to talk to each other and exchange knowledge and views. Poor in the feedback and the discussions with the Microsoft Program Managers and their management. This goes on from sun rise to sunset. It pays to come early and stay an extra day. It opens time for more meetings and discussions in and around Redmond.

image

The end result is a truck load of information and impressions we need to parse. That can take some time. And we need to filter our conclusions for our management. The content of the MVP Summit and all talks around it are strictly NDA. The insight and ideas we harvest from that we can leverage, but we cannot expose the information.

On Microsoft’s side they get a reality check, open and honest feedback, they get our opinions and ideas. They learn about our successes and challenges in the real world. If that was not helpful to them they wouldn’t want us to show up on their campus disrupting their work week.

To me it’s also a reality check. What am I doing. How am I doing that and why? Even more importantly where am I doing this things and is what I use the best choice. It show my own strengths and weaknesses. That’s valuable as well.

Well the good news is that judged by some requests and opinions of my peers I’m an valued expert and architect. I do have some weaknesses but I’m on track to address those. The balancing act here is that we have to avoid wasting time on dying opportunities that are sill needed but are heading down hill fast. Not as much due to the technology being obsolete or no longer needed but mostly due to politics and a bad understanding on how to deliver IT cost effective and efficiently. The amount of self inflicted wounds and pain can be shockingly high. The trick is to avoid those projects as that’s wasted time, time that should be spent on moving forward. Sometimes this looks like the nineteen nineties all over out there.

One thing is very clear. Those that seek a single solution, a one size fits all approach, just for the sake of simplicity or perceived economies of scale will fail. A bipolar approach without a place for the vast amount of “stuff” in between, let alone a realistic and sound technical plan to integrate it all are going to fail. Ask any plumber Winking smile. Learn how to think independently and don’t grow too dependent on industry analysts. Do what’s right for your needs.

In Place upgrades of cluster nodes to Windows Server 2016

You will all have heard about rolling cluster upgrades from Windows Server 202 R2 to Windows Server 2016 by now. The best and recommend practice is to do a clean install of any node you want to move to Windows Server 2016. However an in place upgrade does work. Actually it works better then ever before. I’m not recommending this for production but I did do a bunch just to see how the experience was and if that experience was consistent. I was actually pleasantly surprised and it saved me some time in the lab.

Today, if you want to you can upgrade your Windows Server 2012 R2 hosts in the cluster to Windows Server 2016.

The main things to watch out for are that all the VMs on that host have to be migrated to another node or be shut down.

You can not have teamed NICs on the host. Most often these will be used for a vSwitch, so it’s smart and prudent to note down the vSwitch (or vSwitches) name and remove them before removing the NIC team. After you’ve upgraded the node you can recreate the NIC team and the vSwitch(es).

Note that you don’t even have to evict the node from the cluster anymore to perform the upgrade.

image

I have successfully upgrade 4 cluster this way. One was based on PC hardware but the other ones where:

  • DELL R610 2 node cluster with shared SAS storage (MD3200).
  • Dell R720 2 node cluster with Compellent SAN (and ancient 4Gbps Emulex and QLogic FC HBAs)
  • Dell R730 3 node cluster with Compellent SAN (8Gbps Emulex HBAs)

Naturally all these servers were rocking the most current firmware and drives as possible. After the upgrades I upgraded the NIC drivers (Mellanox, Intel) and the FC drivers ‘(Emulex) to be at their supported vendors drivers. I also made sure they got all the available updates before moving on with these lab clusters.

Issues I noticed:

  • The most common issue I saw was that the Hyper-V manager GUI was not functional and I could not connect to the host. The fix was easy: uninstall Hyper-V and re-install it. This requires a few reboots. Other than that it went incredibly well.
  • Another issue I’ve seen with upgrade was that the netlogon service was set to manual which caused various issues with authentication but which is easily fixed. This has also been reported here. Microsoft is aware of this bug and a fixed is being worked on.

 

.