Upgrading a Hyper-V R2 Cluster to Windows 2008 R2 SP1

For all you people waiting to roll out Windows 2008 R2 SP1 to your Hyper-V cluster here’s a quick screenshot rich run through of the process. Some people claim you need to shut down the cluster services and shut down the guests but this is not the case.  You can do a rolling upgrade and your guests can stay on line on the other nodes, just use live migration to get them there. Now I do recommend to upgrade all the nodes tot SP1 as soon as possible and not staying a mixed Windows 2008 R2 / Windows 2008 R2 SP1 situation in your cluster. But this mixed situation makes upgrades of the nodes in the cluster possible without any down time for the guests (if you have live migration), which is the aim of having a high availability cluster.

Walk Through

Live migrate all the guests from the node you wish to upgrade to SP1. Make sure the host is fully patched and disable any antivirus services if you are running any. I always reboot the node before a major upgrade to make sure we have the server in a clean state with  no lingering reboots waiting  or processes can cause issues.

Navigate to the service pack 1 file for Windows 2008 R2, it’s called windows6.1-KB976932-X64.exe and start it up:

You’ll have to accept the license terms:

And then the preparation process starts:

It is now ready to start the upgrade and yes we want it to reboot automatically when needed:

The upgrade process takes a while (about 17 minutes on my lab servers):

 

When it’s done it will reboot and bring you back to the logon screen. Multiple reboots might be needed to complete the upgrade process depending on what’s running on your server. In this case, we are dealing with dedicated Hyper-V nodes.

View when connected to the console

View when connected via RDP

After logging on you are greeted with this window:

And yes this is indeed the case

Reboot included the entire process took about 22 to 23 minutes. In the setup event log you’ll find these messages:

  • Initiating changes for package KB976932. Current state is Absent. Target state is Installed. Client id: SP Coordinater Engine.
  • Package KB976932 was successfully changed to the Installed state.

Note: if an extra reboot is required you’ll see an extra entry in between these stating: A reboot is necessary before package KB976932 can be changed to the Installed state.

When you have a cluster with nodes running both W2K8R2 TM and W2K8R2 SP1, mixed situation so to speak,  you’ll see the following notification in the cluster events:

You can live migrate the guest from the next node to the node already upgraded to SP1 and than repeat the process. You keep doing this until all your nodes are upgraded.

As a final recommendation, I would suggest waiting until you get the SCVMM2008R2 SP1 bits is you use this product before you upgrade your clusters especially when using this with SCOM2007R2 PRO Tips. Otherwise, you don’t need to wait just realize that until you have SP1 for SCVMM2008 R2 you won’t be able to use the new functionality for Hyper-V. In production, I would not recommend using the RC1 for this.

Please do not forget to update your guests with the new SP1 version of the Hyper-V Integration Components. This is needed to be able to use the new features like Dynamic Memory & Remote FX. The Windows 2008 R2 RTM version of the Integration Components is  6.1.7600.16385:

image

You can do this using Hyper-V Manager through selecting “Insert Integration Services Setup Disk”  and running the setup, this will require a reboot.

Click to start the upgrade process.

It will ask to upgrade or repair the previous version:

Work in progress:

Done and asking for a reboot:

SCVMM2008R2 can also be used, here you shut down the guest before updating the virtual guest services as it’s called in SCVMM2008R2. It can be annoying that the nomenclature differs. The good thing here is that you can upgrade multiple guest using VMM2008R2. Hans Vredevoort did a blog post on this here: http://www.hyper-v.nu/blogs/hans/?tag=windows-server-2008-r2-sp1.  After the upgrade you can see that the version of the Integration Components  for Windows 2088 R2 SP1 is  6.1.7601.17514:

Windows 2008 R2 / Windows 7 is RTM today (for real this time)

After an initial false start (https://blog.workinghardinit.work/2011/01/14/windows-2008-r2-windows-7-sp1-rtm-today/) Window 2008 R2/Windows 7 Service Pack 1 has been officially RTMed today . SP1 brings Dynamic Memory& RemoteFX to hyper-V virtualization. I’m probably doing a SQL Server virtualization project and thus I’m very interested in the ability to disable NUMA spanning (watch Ben Armstrong’s Tech Ed 2010 presentation here) http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/06/10/talking-about-dynamic-memory-the-movie.aspx) when beneficial to do so . Which is good news. Until now SQL Server Hyper-V host where better of using machine with lesser CPU sockets & and SSL server VM’s that don’t consume more RAM than that CPU socket can address directly to avoid this.  Until now, for the  environment at hand, I’m leaning to virtualizing SQL Server on it’s own Hyper-V cluster for that reason. It will have to be confirmed in a test environment to see how big the impact is. Systems do differ and get better every year. Perhaps I’ll get back to that subject later. Anyway The bits should be on TechNet/MSDN on February 16th 2001 and available to the general public on February 22 2011. Read the announcement by Microsoft here Windows Server 2008 R2 and Windows 7 SP1 Releases to Manufacturing Today

Shameless Plug For Mastering Hyper-V Deployment By Aidan Finn

In October 2010 Aidan Finn (MVP) his book “Mastering Hyper-V Deployment” was released and in November three copies of this book landed on my desk. I bought them (pre order) via Amazon. Nope I did not get them as a gift or anything. Why Three? Well that’s the number of people I wanted to get up to speed about Hyper-V and virtualization management and operations in a Microsoft environment.

His book takes you along a journey through a Hyper-V project that will teach you about virtualization in all it’s aspects. It also touches on many supporting technologies and products such as System Center Virtual Machine Manager 2008 R2, System Center Essentials 2010, Data Protection manager 2010 and System Center Operations Manager 2007 R2. No one book can be the only source of knowledge and understanding, but using this book as a start for both new and experienced IT Pros to learn about virtualization with Hyper-V will give you the best possible start. Consider it going to an Ivy league college on a scholarship paid for by Aidan’s experience and hard work. The subsidized tuition fee is the price of the book.

We feel a bit sorry that Aidan only got one copy so we made a group picture of the gang of three on the desk of our newest team member. He got a copy of the book together with 4 recycled PC’s and a TechNet subscription to build a lab.

If you know people who want or need to learn about Hyper-V, you’d do well to make sure they get this book and have them set up a lab to play with the technologies. Those efforts will pay off big time when they implement their solutions in the wild. If Ireland is doomed it won’t be because of smart & hardworking Irish IT professionals like Aidan. You see when you design, build and support IT solutions that your customers depend on 24/7 you can not hide behind false promises, you can’t fake away from the fact when “stuff” doesn’t work or hide behind vast amounts of papers & documents void of any substance. Nope, you are responsible for everything and anything you build. Aidan backed and supported by some very knowledgeable colleagues has made that burden a bit lighter for you to bear with this book. Aidan’s blog lives here: http://www.aidanfinn.com/

Solving USB License Dongle Needs & COM Port Needs In Virtual Machines – RemoteFX USB Support Won’t Replace USB Over Ethernet

The Issue

One of the biggest showstoppers people come across when virtualizing servers is often software license dongles. A lot of software vendors have decided that such measures are needed to protect ‘m from piracy. I don’t think it works. I might prevent the occasional unlicensed use but it has never stopped a determined “pirate” from cracking & hacking the dongles or the software and offering that solutions on the dark side of the internet. The only cookies they have are poisonous 😉

But short of all the stupid issues paying customers will have with license dongles, when it’s time to virtualize the servers you’re in trouble. There is no USB device on a virtual machine to stick that dongle into. Some quality desktop virtualization solutions like VMware workstation do offer USB support but the heavy duty server production hypervisors do not. So now there is an extra issue to solve. The solution to that problem can be solved USB over Ethernet software. When we start talking about physical COM port mapping (SMS Gateways anyone and other “esoteric” engineering toys …) it’s the same issue. But what’s discussed below is also valid for that problem. The companies have software for that as well.

The Solution

When you need to virtualize server applications that need a locally attached dongle you need to start looking for USB over Ethernet solutions that are reliable, support x64 bit, have signed drivers and support your Operating Systems. Normally you pay for the number of servers and USB device connections for which you get a license key. The client software is free and unlimited.

I’ve used both Fabulatech and KernelPro. But now I exclusively use KernelPro. FabulaTech version 4.1.1 was rock solid but ever since version 4.2/4.3 & 4.4 Beta they have brought us nothing but “Blue Screen of Death” hell either simply in production or during reboots (server grade hardware, DELL, HP). They really dropped the ball in a huge way It also had issues with clean uninstalls for upgrades etc. We just had to cut our losses after many months of frustration and troubleshooting as we were bleeding money and losing valued customers, not to mention our reputation. We now implement KernelPro USB over Ethernet version 2.5.5 as the solution of our choice. It works very well and we just never had an issue with that product. In the Hyper-V guest you install the client software and make sure to allow the needed ports in the client firewall configuration. For a license dongle you still need the dongle drivers to be installed but once that done you’re golden. To make sure server reboots do not interfere with the licensing I configure the USB over Ethernet software to automatically connect the correct USB device to the correct client. To prevent unwanted client connections you can use a IP filter list to allow or disallow client connections and disable USB devices for sharing. When using DNS names be aware of DNS Client Caching etc. when you change the IP address of a client and such. During fail over testing that might trip you over. I your application supports multiple dongles you can even create fault tolerant USB over Ethernet setups. Now try to sell that to a client: “Hello, I’m the architect of you highly available USB over Ethernet License Dongle solution for your Hyper-V farm”. Been there, done that!

Due Diligence

As I’ve written in a previous blog post “Perversions of IT: License Dongles” (https://blog.workinghardinit.work/2010/03/29/perversions-of-it-license-dongles ) you need to be aware of the fact that you need to manage the Ethernet over USB environment very carefully. Make sure the versions of the server software and the client software are the same on all nodes. We’ve seen network connectivity loss when the versions don’t match up, even if the software didn’t complain about different versions. Always make sure they are running the same version. Don’t just randomly update. For this reason I always disable the check for updates.

You need to test its stability, have some extra hardware and extra dongles for testing as to confirm your dongles respond well to this type of setup. We can’t afford to bring down production environments with USB over Ethernet software “upgrades of faith”. With some clients the license dongles become a virtualization show stoppers due to such risks. Talking about technology debt a cheap piece of plastic license protection is preventing paying customers from virtualizing. How do you think they feel about you as a software company. You’ll find out when they find an alternative without a dongle 😉

RemoteFX USB Support to the rescue?

When Windows 2008 R2 SP1 was announced there was also a lot of talk about support for USB devices via RemoteFX USB . I was very happy with that because it’s a very useful and handy feature. Initially I was also optimistic about the fact that it might help with the license dongle issue. But as the name says, it very much geared to offer support for VDI environments. That means you’ll be able to hook up cameras, license dongles for interactive user apps so you can run them on your virtual desktop, USB rocket launchers (http://www.youtube.com/watch?v=EmZ-QKglyrc ) and the works.

Now for server applications there is no option provided for mapping the USB dongle on a parent partition to a child partition permanently. So basically for such needs , and in my neck of the woods I have a bunch of those, there is no built in solution. Software running as a service, License software running a couple of dongles and offering network licenses for applications will still need a USB over Ethernet solution. One little positive thing is that I heard rumors that one software vendor is dropping the need for a dongle. On small step but we’ll call that progress anyway. I hope the virtualization push forces a lot of others to do likewise and forget about license dongles. Typically for such companies that will be very late in the game. By the time they acknowledge the needs and realties in the field most customers have dumped that vendor or work around the problem on their own time and at their cost. So a vendor solution is always late. When they have it the industry progress has already led to another problem with their software due to bad choices they made in the past. Such is life. Still I have one advice for software companies: become good or get out!