Cluster Operating System Rolling Upgrade Leaves Traces

Posted on by
5

Introduction

When you perform a cluster OS rolling upgrade of Windows Server 2012 R2 cluster to a Windows Server 2016 Cluster you’ll have two options.

1. You evict the nodes, one after the other, perform a clean OS install and join them to the existing cluster.

2. You do an in-place OS upgrade of the nodes (no need to evict the nodes, you can if you want to). I tested this and blogged about it in In Place upgrades of cluster nodes to Windows Server 2016  

Both of these give you the benefits that you can keep your workloads (Hyper-V, SOFS, SQL Server) running and you don’t have to create a new cluster to do so. The moment you have Windows Server 2016 Nodes added to an existing Windows Server 2012 R2 cluster you are running in Mixed mode. Until all your nodes have been upgraded to Windows Server 2016 will remain running in mixed mode.

Illustration showing the three stages of a cluster OS rolling upgrade: all nodes Windows Server 2012 R2, mixed-OS mode, and all nodes Windows Server 2016

When there are only Windows Server 2016 nodes you can decide to also upgrade the cluster functional level.  This enables all the new capabilities in Windows Server 2016 Failover Clustering and also means you cannot go back to a Windows 2012 R2 cluster anymore. So, only take this step after a final validation of all drivers and firmware to make sure you don’t need to go back and you’re ready to fully commit to a fully functional Windows Server 2016 Failover Cluster.

A cluster operating system rolling upgrade does leave some traces, but that’s OK. Let’s take a look. 

This is what a get-cluster against Windows Server 2016 that was upgraded from Windows Server 2012 R2 looks like.

image

As you can see the cluster functional level is 8 and not 9 yet. This means that we have not yet run the Update-ClusterFunctionalLevel command on this cluster yet. Which still allows us to roll back all the way to a cluster running only Windows 2012 R2 nodes. The ClusterUpgradeVersion has a value of 3.

We now execute the Update-ClusterFunctionalLevel command and take a look at Get-Cluster again.

image

As you can see we are now at cluster functional level 9 which enables all the capabilities offered by Windows Server 2016 Failover Clustering. The cluster Upgrade version is 8. That’s the previous cluster functional level we were at before we executed Update-ClusterFunctionalLevel.

Note that both properties ClusterFunctionalLevel and ClusterUpgradeVersion are only available with Windows Server 2016. You will not find it on a Windows Server 2012 R2 or lower cluster. If you run this command from Windows Server 2016 against a Windows Server 2012 R2 cluster both properties will be empty. If you run it on a Windows Server 2012 R2 host against Windows Server 2012 R2 or lower and even a Windows Server 2016 cluster these properties are not even there. The commandlet is older on those OS versions and didn’t know about these properties yet.

What about if you create a brand-new cluster, perhaps even on freshly installed windows Server 2016 Nodes? What does ClusterUpgradeVersion have as a value then? Well it’s also 8. In the end, there is no difference between an in-place upgrade Windows Server 2016 cluster and a cleanly created one. So where are those traces?

Cluster Operating System Rolling Upgrade Leaves Traces

What gives a rolling upgrade away is that in the registry, under the HKLM\Cluster the OS and OSVersion values are not updated (purple in the picture below). This is a benign artifact and I don’t know if this if on purpose or not.  I have changed them to Windows Server 2016 Datacenter as an experiment and I have not found any issues by doing so. Now, please don’t take this as recommendation to do so. The smartest and safest thing is to leave it alone. These are not used, so don’t worry about them.

image

But even if you would change those values a cluster resulting from of a cluster operating system rolling upgrade still has other ways of telling it was not born as a Window Server 2016 Cluster.

Under HKLM\Cluster (and Cluster.0) you’ll find the value CusterFunctionaLevel that does not exist on a cleanly installed Windows Server 2016 Cluster (green in the picture above). As you can see this is a Window Server 2016 cluster running at functional level 9.

There is even an extra key OperatingVersion under HKLM\Cluster that you will not find on a cleanly installed cluster either. It also has a Mixed Mode value under that key which indicates whether the cluster is still running in mixed mode or not.

image

Here is a screenshot of newly installed/created Windows Server 2016 cluster. No ClusterFunctionalLevel value, the OS and OSVersion Values are correct and there is no OperatingVersion key to be found.

image

What if you don’t like traces?

First of all, these traces are harmless. One thing you can do if you want to weed out all traces of a rolling upgrade (as far as the cluster is concerned) is to destroy the cluster and create one with the same CNO (and IP address if that was a fixed one). This might all be a bit more involved when it comes to CSV naming and other existing resources but then these remnants will be gone in a supported way. Now this does defeat one of the main purposes of this feature: no down time. The operating system itself might also contain traces if you did in-place OS upgrades but the cluster will not. Just adapting OS/OSVersion, ClusterFunctionalLevel and deleting the key OperatingVersion from HKLM\Cluster (and HKLM\Cluster.0) are not supported actions and messing around in the cluster registry keys can lead to problems, so don’t! The advice is to just leave it all alone. Microsoft developed cluster operating system rolling upgrade the way they did for a reason and by leaving things as Microsoft has set or left them will make sure you are always in a fully supported condition. So, use it if it fits the circumstances & you comply with all the prerequisites. Look at these traces a flag of honor, not a smudge on your shining armor. When I see these artifacts, I see people who have used this feature to their own benefits. Well done I say.

Learn more about the Cluster OS Rolling Upgrade process

Next to my blogs like First experiences with a rolling cluster upgrade of a lab Hyper-V Cluster (Technical Preview) and In Place upgrades of cluster nodes to Windows Server 2016 there are many resources out there by fellow blogger and Microsoft. A great video on the subject is Introducing Cluster OS Rolling Upgrades in Windows Server 2016 with Rob Hindman, who actually works on this feature and knows it inside out.

An important thing to keep in mind is that this can be automated using PowerShell or by leveraging SCVMM for orchestration for example. 3rd party tools could also support this and help you automate this process in order to scale it when needed.

Finally, the official documentation can be found here Cluster operating system rolling upgrade

Hyper-V Amigos Showcast Episode 14: RemoteFX & DDA

Posted on by
Reply

Carsten and I dove into our labs and played around with RemoteFX and Discrete Device Assignment in Windows Server 2016 Hyper-V and RDS. This resulted in the Hyper-V Amigos Showcast Episode 14: RemoteFX & DDA.

Some background on RemoteFX & DDA

I’ve discussed the new capabilities in previous blog posts such as https://blog.workinghardinit.work/?s=DDA&submit=Search  and RemoteFX and vGPU Improvements in Windows Server 2016 Hyper-V. But here the Hyper-V Amigos talk about it for your benefit and enjoyment. I for one know we had a ton of fun. Microsoft only VDI solutions are really taking off both on-premises and in Azure in cost conscious environments that still need good performance. I think we’ll see an uptake of such deployments as Microsoft has made some decisions and added some features to make this more feasible.

Hyper-V Amigos Showcast Episode 14: RemoteFX & DDA.

Click this link or the image below to watch Hyper-V Amigos Showcast Episode 14: RemoteFX & DDA

image

There’s a bit of a learning curve associated with using DDA in Windows Server 2016. You’ll have to get acquainted with how to do it and put it to the test in labs and POCs. Do this before you even start thinking about designing production ready solutions. Having a good understanding on how it works and behaves is paramount to success.

Enjoy!

Project management is mission critical

Posted on by
2

Project management is mission critical

Just like analysts, project managers, are to provide added value and will play a crucial role in the success or failure of a project. So when you use them you have to do it right. If not, you are adding a force multiplier to project failure. In other words you are committing self inflicted harm or sabotage.

Work, Workplace, Office, Computer, Desk, Laptop

Picture courtesy of Max Pixel – http://maxpixel.freegreatpicture.com/Computer-Work-Office-Laptop-Desk-Workplace-1515801

Say what?

The above might seem obvious. Project management is mission critical but very often project management is left to the run off in our labor pool. This gives the profession a bad name. Even worse, by association, it ruins the reputation of the people or organizations that hire or engage them. Body shopping shop and “rent a body” outfits benefit from the idea that project management is a thirteen in a dozen job that translates into overpriced meeting transcriptions and carbon fiber Outlook calendar “automation”. In reality the customer is of way better with a real good secretary and people who know how to organize a meeting.

Project management done wrong doesn’t add any value to the project. On top of that it causes expensive and annoying overhead where the death weight of the PM becomes a nuisance at best. Potentially it becomes a fatal burden.

The 3 critical skills of a project manager

People skills

People respond to you in a way that depends on you threat them. They are a mirror. If you step on them for your own agenda or to protect your billable hours it will be noted and never forgotten.

Diplomacy

Some people state that diplomacy is the fine art of telling people to go to hell and feel good about it. Well, maybe. There are two cases for diplomacy in project management when the situation or relationships are bad and cannot be fixed:

  • Neutralize saboteurs, idiots, bad politics and hidden agenda’s in the most eloquent and pleasant way possible in order not to jeopardize the final outcome of the project.
  • It’s what you do to keep the above occupied before you get rid of them if that’s an option. In that regards PMs are morphine. They make the bad elements believe things will be fine, to make sure it painless while they’re terminated.

A PM needs to fight and defend the people they work for. If you do not agree with the people you work with or cannot agree with them because they are wrong, you have to walk away. If not, you’re just there for the paycheck at the expense of others and the project.

Subject Matter Expertise

Don’t bother sending a PM that’s not a subject matter expert. It’s a waste of time, effort and money. If the PM doesn’t understand the area of expertise, can’t learn fast and doesn’t trust the team experts it will never work out. Perhaps even worse is that many times they don’t care and the subject matter is seen as the dough to be put through the generic project management cookie cutter.

It ruins moral to be led by an incompetent, uncaring PM that is unable to truly weigh the pros and cons to the benefit of the result. In such cases they are dead weight, holding back progress and ideas which results in a very predictable failure. You’ll see burn outs, depression or people who walk. in the end it all signals the same. They don’t want to or cannot deal with the crap anymore. Results do matter and in the end no one buys a journey or services. We pay for results.

Conclusion

Do it really well or cut the bullshit and get on without one, the results without a PM are always better than with a bad one. I know it’s a dog eat dog world out there and that’ just how things flow but when a dog comes to eat your lunch you don’t play victim to circumstances. You act or you perish. A quick search reveals many are having issues with project management instead of being helped by it. It doesn’t need to be that way.

I’m a Veeam Vanguard 2017

Posted on by
3

I’m a Veeam Vanguard 2017 I learned tonight

I came home tonight after a long day at work. The day was one of trying to solve some weird remote client issues, helping colleagues, supporting the devs, etc. There was also some fun whilst teaching some PowerShell skills to a colleague. It keeps them motivated and it also showed him future path were thing can and are going. Which reminds me of the good feeling being part of the Veeam Vanguard program gives me. Speaking of which …

What put a huge smile on my face tonight was an e-mail confirming I was invited to the  Veeam Vanguard program again in 2017. Excellent news! And yes I did accept. What did you think!?I’m a Veeam Vanguard 2017

Being a Veeam Vanguard

So yes, I’m a Veeam Vanguard 2017. The Veeam Vanguard program is to me is about sharing, learning and growing through interaction with some of the finest techies any company out there could ever hire. It is full of hard core, highly skilled and experienced technologists from around the globe. They range from intensively hands on types deep in the trenches to real architects & strategists.

These are people that elevate me, teach me, educate me and, every now and then, make me feel very humble due to the depth and breadth of their knowledge. While I have my strengths, I sometimes feel out of my league among them. But that’s OK. Not being the smartest guy in the room and group means I’m learning and advancing. It makes it interesting and the diversity in skills and technologies make for a better stronger group that can deal with wide variety of challenges.

Everybody contributes to help each other, customers and clients which is good for us, for everyone using Veeam products and for Veeam. Veeam builds great software, has stellar support and delivers the value I look for in an ISV.

So to Veeam, thank you for the trust in me, my skills and professional abilities to make me a Vanguard in 2017. I’m happy and proud to be one.