The Hyper-V Processor Relative Weight

Introduction

Hyper-V offers 3 ways of managing or tweaking the CPU scheduler to provide the best possible configuration for certain scenarios and use cases. The defaults normally work fine but of certain conditions you might want to tweak them for the best possible outcome.  The CPU resource controls at your disposal are:

  • Virtual machine reserve  – Think of this as the minimum CPU “QoS”
  • Virtual machine limit – Think of this as the maximum CPU “QoS”
  • Relative Weight – Think of this as the scale defining what VM is more important.

Note that you should understand what these setting are and can do. Threat them like spices. Select the ones you need and don’t overdo it. They’re there to help you, if needed you can leverage all three. But it’s highly unlikely you’ll need to do so. Using one or two will server you best if and when you need them.

In this blog post we’ll look at the relative weight.

Relative weight

Relative weight is a relative number between 1 and 10000 that you can assign to a virtual machine. This determines the relative importance of a virtual machines CPU resources in regards to other virtual machines. So it’s not a % or number of cycles, it’s just a arbitrary weight. By default this is set to 100.

image

You need to come up with a scale and stick to it. 100, 200 and 300 for low, medium and high important virtual machines is a good example. You could also create 10 “classes”  1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000, 5500. This leaves room to even create even more (lower, in between and higher).

Note that as long as there are sufficient CPU resources on a host the relative weight does not come into play. It really doesn’t matter whether a virtual machine has relative weight of 1000 versus 5000 at that time. They both get whatever they need as there’s plenty to go around.

Relative weight kicks in when the demand is higher than the availability on a physical host. When you have left all the virtual machines at the default of 100 they will all get an equal share. But when you’ve set virtual machines with a higher relative weight these will be getting a higher share of the available CPU cycles.

Use Cases

Not all virtual machines are created equal. In reality some workloads are more important than others. This might be development and test versus production or high priority workloads versus lower priority workloads. The lower priority workloads are the once that you care about less when there is contention for CPY cycles. Or workloads where less CPU cycles and slower response times don’t make a real difference.

Another use case might be on your developers or lab host where you have a CPU sensitive workload you give a much higher weight and leave the others at the default of 100.

To make sure the high priority workloads or those that really depend on more CPU cycles being delivered fast don’t have to play second fiddle to those that don’t have those needs we use relative weight. It’s very flexible and only kicks in when needed, so there is no waste or inefficiency there.

Limitations

The biggest limitation is in the name. It’s all relative. Where as reserve or limit give you a minimum and a maximum respectively, the relative weight only defines what virtual machine more important than another in regards to CPU cycles. So some virtual machines get more than others but that might not be enough. It’s all about balance between virtual machines, not guaranteed minima or maxima.

You need to agree on a standard within the company to define weight. If everyone starts using a different scale you’re in trouble.

Let’s take one admin who uses 100 for less important virtual machines, 200 for standard virtual machines and 300 for the most important ones. That’s all great when he’s the only one defining the settings and when he does so consistently on all nodes/ cluster for all VMs. In that case all is well even when VMs move around between hosts or between clusters. But what happens when many admins use different “scales”. Well it’s a mess and the behavior won’t be what you want when your colleague used 1000, 2000 and 3000 respectively for the same definition. It’s also smart to not use 100, 101 and 102. leave some margin for adding a category when needed.

Conclusion

This is one handy tool to have at your disposal and I tend to use it to proactively set a higher weight for very important VMs. Even in an environment where there are no predefined categories or know minima this allows me to tell Hyper-V that, if there ever is contention for CPU cycles, the virtual machines with a higher weight are the one to serve a bigger share of the limited resources.

The Hyper-V Processor Virtual Machine Reserve

Introduction

Hyper-V offers 3 ways of managing or tweaking the CPU scheduler to provide the best possible configuration for certain scenarios and use cases. The defaults normally work fine but of certain conditions you might want to tweak them for the best possible outcome.  The CPU resource controls at your disposal are:

  • Virtual machine reserve  – Think of this as the minimum CPU “QoS”
  • Virtual machine limit – Think of this as the maximum CPU “QoS”
  • Relative Weight – Think of this as the scale defining what VM is more important.

Note that you should understand what these setting are and can do. Threat them like spices. Select the ones you need and don’t overdo it. They’re there to help you, if needed you can leverage all three. But it’s highly unlikely you’ll need to do so. Using one or two will server you best if and when you need them.

In this blog post we’ll look at the virtual machine reserve.

Virtual Machine Reserve

The virtual machine reserve is a minimum CPU guarantee. This is configured at the virtual machine level for all vCPUs of that virtual machine. This means a couple of things. The reserve is guaranteed for the virtual machine that has it configured. This means that when there is contention for CPU resources the reserve is enforced. When there is no reserve configured are left to compete for the available resources, they don’t have any “SLA” minimum guaranteed CPU cycles however

image

The virtual machine reserve is configured as the minimal % of CPU usage we’ll guarantee a virtual machine to get when there is contention. 0% reserve means that the virtual machine reserve is disabled. This is the default.

So how is this reserve accomplished? When a virtual machine is started there is a check if the virtual machine reserve can be met. If not, the virtual machine will not be started. This is also the case when you want to quick or live migrate a virtual machine. You cannot migrate a virtual machines to a host that cannot deliver the reserve.

The reserve is guaranteed. When you have started enough virtual machines with a reserve to consume 100% of the host CPU resources you cannot start another virtual machines that has a reserve configured. This is even the case if all the running virtual machines who together have reserved 100% of the host CPU resources are completely idle. That’s the cost of reserving CPU resources.

Also note you can mix virtual machines with and without a reserve configured. That’s perfectly fine.

Below is an extreme example. This single virtual machines has 56 vCPU configured with a 100% reserve. This means that this virtual machines will only start on a host that has no other virtual machines with a reserve running. Virtual machines without a reserve will run but when this virtual machines requires the minimum to be met they’ll lose their CPU cycles.

image

So how does this work

When the virtual machines are running and contention for CPU resources materializes on the host than the reserve will kick in and guarantee the virtual machines get there guaranteed minimum. If there is no contention this isn’t enforced and all virtual machines can grab CPU resources as long as they are available. Only when a reserve can’t be met due to lack of resources the CPU scheduler intervenes. This means there is no waste by not allowing other virtual machines to use what is not needed at any given moment by others. Yes virtual machines can use more than their reserve if it’s available and other virtual machines don’t have a need for their reserved CPU cycles. But the moment there is contention, the reserve is guaranteed. That’s why you cannot reserve more that 100% in total. This however makes things a bit more complex in real life. A reserve is not enforced when not needed but it does have to be there. This means that if you reserve a 30% for a virtual machine with 56vCPU you can’t even reserve 80% for a virtual machine with 8 vCPU and run those together.

image

Even when you have 56 cores left with 70% CPU unreserved you cannot find one single CPU with 80% free, let alone 8 of them. This means you cannot start such a virtual machine when the first one is running and vice versa. So that’s another caveat you need to consider.

Use Cases

The primary use case of virtual machine reserve is to enforce an SLA for a virtual machines compute needs. When a DBA demands 100% of the 8 vCPUs his SQL Server virtual machine has configured you can set the reserve to 100%, which is 14*% of all CPU resources on the host. On our 56 core host that means at least 8 CPU have to have 100% of resources free or that SQL Server virtual machine won’t even start as the reserve can’t be guaranteed. imageWhen it’s running and there is no contention for resources other virtual machines can grab whatever they need. The CPU scheduler will only take resources away from the other virtual machines if it’s needed to guarantee the reserve configured for the SQL Server virtual machine.

Some people get creative and use the virtual machine reserve to allocate 100% of all host CPU resources to all virtual machines running on a host. This means all resources are taken and they’ll need to rebalance the reservation if you want to add more virtual machines with a certain reserve. This only works if you can enforce all the virtual machines are created with a reserve. If not some of the new virtual machines might not get any CPU cycles when here is a lot of contention.

Limitations

It can become high maintenance. When people set up CPU reserves for all virtual machines (let’s say 14 of them each with 2 vPCU)  on a host with 2*14 cores. That means we can give each virtual machine a reserve of 10%. They all get 2*10% of 28 cores. 

When you now add a virtual machine with a reserve to the host you’ll need to reset the reserves to accommodate 11 virtual machines. As said this can be used to make sure no VM get’s deployed and started without you noticing (some one will complain when a VM doesn’t start) but this is tedious to manage.

Also note that this game of percentages plays per host but those can be part of a cluster. So when you do this on all nodes on a cluster things will go wrong if you have reserved 100% of the CPU resources on all nodes, live migration and failover will run into issues as extra virtual machines can’t start on nodes that haven’t got sufficient CPU resources left. You need to account and plan for for this

Conclusion

Virtual machine reserve is a powerful tool to have when needed. You do need to understand its behavior very well and take that into behavior for startup, quick and live migrations.

If you do that and use it wisely it can help you achieve SLA’s and the best possible guaranteed performance when needed.

Live Export a Running Virtual Machine or a Checkpoint

A remarkably little known feature in Windows Sever 2012 R2 (and Windows 8.1)  is the ability to export one or multiple running virtual machines.

image

You just select right click in the Hyper-V manager and select Export from the context menu and follow the wizard to select an export location. Easy. This is also possible via PowerShell so you can automate this. The result is a VM you can import which gives you a copy of the original virtual machine in a saved state, at the point in time that you exported it.

More people seem to know about the capability to export a checkpoint of a running virtual machine, not so many of the capability to export a running VM itself. I noticed this because some people figured the latter was a new feature in Windows 2016. No it’s not. We’ve had this option since Windows 8.1 and Windows Server 2012 R2.

image

So why even have the option of exporting a checkpoint of a running VM? Because this enables you to have exports from various points in time, which is pretty cool and handy during test and development and trouble shooting or lab work. As a standard checkpoint has state in Windows Server 2012 R2 I prefer to shut down the VM, create a checkpoint and start the VM again. When I then export that checkpoint I don’t have to worry about the state in the VM at that point in time as it was shut down.

For some workloads this isn’t a big deal bit for some this is not a great experience, hence the fact that checkpoints are “”not supported in production but for test and dev.

In Windows Server 2016 we now have production checkpoints. That means that when we apply such checkpoints we have a consistent state just like when we restore VM from a backup. You’ll have to boot it up after applying the checkpoint, they do not appear running with the state at the time the snapshot was taken. Well, not unless you opt to create standard checkpoints. The reduces the need for me to shut down a VM before I create a checkpoint to export in many cases.

When you export a running VM in Windows Server 2016 you’ll have a copy of it in saved state. Just like you did in Windows Server 2012 R2, no change there. When you import that you’ll have a VM in saved state that you need to start up. If you want an application consistent copy, create a production checkpoint first and export that one.

So there you go. The feature to live export a running virtual machine was here before and it’s still here. The real extra capability with live exports comes from leveraging the live export of a checkpoint of a running virtual machine and the fact that we now have production checkpoints.

E2EVC 2016 Dublin

So after 4 days in London with the Veeam Vanguards I’m hopping over the Irish sea tonight towards Dublin to attend and present at the Experts 2 Experts Virtualization Conference (E2EVC) 2016  Dublin.

image

I’ll be presenting on High Availability with Windows Server 2016 Failover Clustering and Hyper-V. These are supported and enhanced by other new or improved capabilities in Windows Server 2016. So there will be a lot to take in. I’ll be around to talk shop before and after the session so if you want to chat, find me around the conference.

E2EVC a boutique community driven conference. It’s small scale and independence makes it a very direct experience in both realism and hard core real world experience in the virtualization ecosystem and related technologies. We encourage anyone to attend and present. No “prima donnas”  – you can speak when you want to as long as you’re willing to do so and share knowledge and insights.  Give it a shot!  I hope to see you there