Monthly Archives: October 2014

Windows Server 2012 R2 Clustering brings improved CSV diagnosability

Posted on by
Reply

Cluster Shared Volumes (CSV) have can go into different redirected access modes for several reasons. Now a lot of people get (or got) worried about seeing “redirected access” in the GUI. Most of the time however this is due to normal operations such as backups or maintenance (defragmentation) not only losing disk access.

To remediate unneeded troubleshooting, sometimes leading to real issues, calls to MSFT support and so forth it was hidden from the Failover Clustering GUI in Windows 2012 R2. OK, so goal achieved but how do we now troubleshoot and view redirected access that might indicate the presence of real issues? The answer to that is the Get-ClusterSharedVolumeState PowerShell cmdlet. It displays the state of the CSVs on a per node basis for a cluster. You’ll see the type of the IO (Direct, File System Redirected and Block Redirected), if it’s completely unavailable  as well as the reason.

This is what the output looks like on a two node cluster where node A has lost it’s storage path or paths (MPIO) to the CSV. You’ll see that both CSV are in redirected access. Not only that but you can see what type (block redirected) and why (no disk connectivity).

image

Pretty neat and clear. I love this functionality by the way and It’s why I’m leveraging 10Gbps Ethernet extensively to make sure that CSV traffic get’s the bandwidth & latency to handle what it has to. If you realize it leverages SMB3 which provides SMB Multichannel and SMB Direct you know it will get the job done for you in your time of need.

While this is happening in the GUI you’ll see this

image

Nothing is going on … it would seem so a bit of monitoring and alerting would be of use here. The good news is finding out what’s up is very straight forward now.

Now there is still a case where you’ll see that the CSV is in redirected access mode and that when you’ve put it in there yourself via the GUI

image

or via PowerShell for maintenance reasons.

image

As you can see the Icon has change to a networked disk one and it states “Redirected Access”. With Get-ClusterSharedVolumeState the output looks like this.

image

You’ll always see warnings in the event logs.

image

So monitor those with SCOM or another tool that suits your taste and you’ll be in good shape to react when it’s needed and you now know how to find out what’s going on.

First experiences with a rolling cluster upgrade of a lab Hyper-V Cluster (Technical Preview)

Posted on by
2

Introduction

In vNext we have gotten a long awaited  & very welcome new capability: rolling cluster upgrades. Which for the Hyper-V roles is a 100% zero down time experience. The only step that will require some down time is the upgrade of the virtual machine configuration files to vNext (version 5 to 6) as the VM has to be shut down for this.

How to

The process for a rolling upgrade is so straight forward I’ll just give you a quick bullet list of the first part of the process:

  • Evacuate the workload from the cluster node you’re going to upgrade
  • Evict the node to upgrade to vNext from the cluster
  • Upgrade (no in place upgrade supported but in your lab you can get away with it)
  • Add the upgraded node to the cluster
  • Rinse & repeat until all nodes have been upgraded (that can take a while with larger clusters)

Please note that all actions you administration you do on a cluster in mixed mode should be done from a node running vNext or a system running Windows 10 with the vNext RSAT installed.

Once you’ve upgraded all nodes in the cluster, the situation you’re in now is basically that you’re running a Windows Server vNext Hyper-V cluster in cluster functional level 8 (W2K12R2) and the next step is to upgrade to 9, which is vNext, no there no 10 yet in server Winking smile

You do this by executing the Update-ClusterFunctionalLevel cmdlet. This is an online process.  Again, do this from a node running vNext or a system running Windows 10 with the vNext RSAT installed. Note that this is where you’re willing to commit to the vNext level for the cluster. That’s where you want to go but you get to decide when. When you’ve do this you can’t go back to W2K12R2. It’s a matter of fact that as long as you’re running cluster functional level 8, you can reverse the entire process. Talk about having options! I like having options, just ask Carsten Rachfahl (@hypervserver), he’ll tell you it’s one of my mantras.

image

When this goes well you can just easily check the cluster functional level as follows:

image

When this is done you can do the upgrade of the VM configuration by running the Update-VMConfigurationVersion cmdlet. This is an off line process where the VMs you’re updating have to be shut down. You can do this for just one VM, all or anything in between. This is when you decided you’re committing to all the goodness vNext brings you.  But the fact that you have some time before you need to do it means you can  easily get those machine to run smoothly on a W2K12R2 cluster in case you need to roll back. Remember, options are good!

Doing so updates VM version from 5 to 6 and enables new Hyper-V features (hit F5 a lot or reopen Hyper-V Manager to see the value change.

image

image

Note: If in the lab you’re running some VMs on a cluster node are not highly available (i.e. they’re not clustered) they cannot be updated until the cluster functional level has been upgraded to version 9.

Win a free ticket to Experts Live 2014

Posted on by
Reply

As you might already know I’m speaking at the Dutch IT community event Experts Live 2014 in the Netherlands. The talk is about “The Capable & Scalable Cloud OS “ where we’ll highlight and show some of the scalable capabilities in Windows Server 2012 R2 when combined with great hardware.

You can find the program at Experts Live 2014 which is very rich in content. There are 7 tracks and over 40 sessions! Chose a track or mix and match to your hearts content between  Microsoft Azure, System Center, Hyper-V, SQL, Windows, PowerShell and Office365. It’s all good.

image

To celebrate the success of the event the organizers have allowed us to give away some free entrance tickets. This is a very nice gift that will allow you to enjoy a full day of learning for free.

So convince me you’re willing to put in the time and effort to learn and we’ll help you do exactly that by making sure you get a free ticket!  Leave a reply to this blog post from Thursday October 9th till Thursday October 16th in which you tell me what blog or blogs of mine you’ve enjoyed most. Leave your name, e-mail, your company and function title so we can arrange things for you. Don’t worry we will not publish these.

There is only one request/condition … if you win a ticket come to the event as a no show means some one else can’t come.

Hyper-V did not find virtual machines to import from the location . The operation failed with error code ‘32784’.

Posted on by
Reply

I got contacted by some people how ran into some issues importing VMs from W2K12R2 Hyper-V into W2K12 Hyper-V. They got bitten by this “little” issue: Importing a VM that is exported from Windows Server 2012 R2 into Windows Server 2012 is not supported

This means you get greeted by

Hyper-V did not find virtual machines to import from the location <folder location>.
The operation failed with error code ‘32784’.

image

No the trick of not exporting the VM but doing an “in place” registration doesn’t cut it. That’s great for W2K8R2 to W2K12 or W2K12 to W2K12R2 but not from W2K12R2 to a lower version. In that way the title of the KB article could be seen as a bit misleading or incomplete, but the contents is pretty clear.

And that’s it. Woeps! What you have 200 VMs on the LUNs form the old cluster you already blew away to build the new one? You do have a tested exit plan for this right? Uh no?

Facepalm Combo

Oh MAN, NOOOOO!

Now if it’s only one or two VMs you can always work around this by creating new VMs using the old VHDXs. This will leave you to deal with networking cleanup inside of the VMs and configuring TCP/IP. PowerShell can help here but in large volumes this remains as serious effort. This is also the time that documentation pays!

Now what if this happens to you when you’re trying to roll back a migration of a hyper-V cluster (revert W2K12R2 to W2K12 for example). Well for one you should have know as you did test all this right? Right?!

What are your other options to roll back other than  the above? From the top of my head and without details?

  • Move back to your old cluster Smile You didn’t already nuke it, I hope.
  • If you have a SAN take a snapshot of the LUNs before you move them to Windows Server 2012 R2 for faster fall back. But beware, if you’re running applications that require some tender loving care in relation to snapshots like Exchange  or Active Directory in those VMs … shutting all VMs down before you create the can help snapshot mitigates issues but is not a full proof approach! “Know thy apps”!
  • A great backup & RESTORE solution to get you back up and running also comes in handy but don’t forget that it requires you to know your apps as well here. Yes, it’s not always just “CLICKEDYCLICKCLICKDONE”
  • Perhaps it’s now time to activate your paused replicas on the DRC cluster or hosts?  You did test this didn’t you?

Now for anyone involved in a migration to Windows Server 2012 R2 there is no excuse not to know this in advance and to test out the new cluster hardware as much as you can. This minimizes the chance you’ll need to fall back. And please test your exit scenarios, really, I mean it.  Also please, you can migrate one LUN/CSV at the time. Try to run the VMs on the first migrated LUN/CSV before you do all the others. That way you can do some damage control.

Now, this is not great but it is what it is and at least now you know before your migrate Winking smile. We’ve also asked MSFT to make falling back a bit less “"involved” in future versions. Perhaps they’ll do that, I’m pretty sure they’ll consider it. And by what we’ve seen in the recently available Technical Preview they did!