Confusing Mellanox Windows PerfMon Counters

Introduction

So you start out doing SMB Direct. Maybe you’re doing RoCE, if so there’s a good chance you’ll be using the excellent Mellanox cards. You studied hard, read a lot and put some real effort into setting it up. The SMB Direct / DCB configuration is how you think it should be and things are working as expected.

Curious as you are you want to find out if you can see Priority Flow Control work. Well, the easiest way to do so is by using the Windows Performance Monitor counters that Mellanox provides.

Confusing Mellanox Windows PerfMon Counters

So you take your first look at the Mellanox Adaptor QoS Perfmon counters for ConnectX series for SMB Direct (RDMA) traffic. When you want to see what’s happening in regards to pause frames that have been sent and received and what pause duration was requested from the receiving hop (or received from the sending hop) you can get confused. The naming is a bit counter intuitive.

clip_image002

The Rcv Pause duration is not the duration requested by the pause frames the host received, but by the pause frames that host sent. Likewise, the Sent Pause duration is not the duration requested by the pause frames the host send, but by the pause frames that host received.

clip_image004

So you might end up wondering why your host sends pause frames but to only see the Rcv Pause duration go up. Now you know why Smile.

Now there were plans to fix this in WinOF 4.95. The original release note made mention of this and this made me quite happy as most people are confused enough when it comes to RDMA/RoCE/DCB configurations as it is.

A screenshot of the change in the original Mellanox WinOF VPI Release Notes revision 4.95

clip_image005

Unfortunately, this did not happen. It was removed in a newer version of these release notes. My guess is it could have been a breaking chance of some sort if a lot of tooling or automation is expecting these counter names.

I still remember how puzzled I looked at the counters which to me didn’t make sense and the tedious labor of empirical testing to figure out that the wording was a bit “less than optimal”.

But look, once you know this you just need to keep it in mind. For now, we’ll have to live with some confusing Mellanox Windows PerfMon counter names. At least I hope I have saved you the confusion and time I went through when first starting with these Mellanox counters. Other than that I can only say that you should not be discouraged as they have been and are a great tool in checking RoCE DCB/PFC configs.

Microsoft Enterprise Agreement Policy Changes

As an existing Microsoft Enterprise Agreement customer you should have already been made aware of the policy changes coming to this type of agreement by Microsoft.
 
image
 
If you haven’t here’s the public blog post where they made it known world wide:

Another step in licensing transformation: new policy and guidance for Enterprise Agreement customers

It’s a good read to get started and gives you some talking points to discuss with your reseller & Microsoft account manager.

One key point to note is that this means that on July 1, 2016 

… the minimum Enterprise Agreement (EA) commitment for commercial customers signing new Enterprise Enrollments or Enterprise Subscription Enrollments will increase from 250 users or devices to 500. Along with this change, we are guiding new commercial customers within the 250 to 499 user or device range to our modern volume licensing solutions: the Microsoft Product and Services Agreement (MPSA) and the Cloud Solutions Provider (CSP).

For those who need some more time to adapt to the new situation and who’s needs don’t get served well by the Microsoft Products and Services Agreement (MPSA) and the Cloud Solution Provider (CPS) offerings there is an option to extend the existing EA for another 3 years. That might well be worth doing.

Stand-alone console in Veeam Backup and Replication v9

Stand-alone console in Veeam Backup and Replication v9

One of the smallish, but significant improvements to Veeam Backup and Replication in version v9 is the introduction of the stand-alone console. That means the GUI is no longer tied to the Veeam Backup & Replication server itself. This is a very welcome improvement.

The default install in a green field scenario does not add the console. That requires a separate install. So if you prefer to do so, you can still mimic your installations to be as they used to be and install the console on the server still. This might be desirable just to have the console in place on the server just in case you need it.

You can specify to use the local host, an IP address or the sever name (FQDN) and choose to either use Windows session authentication when applicable for single sign on or specify the domain & username with a password. It’s pretty flexible.

image

The benefits

What’s the big fuss about this stand-alone console in Veeam Backup and Replication v9Let’s look at what having this stand-alone console enables. Even when you chose to still have the console installed on the Veeam Backup & Replication v9 server itself you’ll enjoy the following new capabilities.

You can install the console on your workstation, laptop, dedicated management server and connect to any Veeam Backup & Replication v9 installation. That could be the one on premises for your company. It could be the ones at your customers. You get the idea. Each admin can have their own console for use with their account or accounts.

An admin can now also easy use multiple accounts or the same account within the same or different environments as long as there is connectivity.

The standalone console allows you to use PowerShell against backup server
remotely, without relying on PowerShell Remoting … A big thank you to Timothy Dewin for pointing this out to me!

You can run multiple instances of the console simultaneously. That  means we can have multiple connections to the same or different VEEAM environments.

image

Normally when every admin is using his or her own account to RDP into the server this is not an issue. But his has actually also been fixed when you do run the console on the Veeam Backup & Replication server itself! You can actually even run multiple instances under the same account or a different account within the same RDP server session to the same or different deployments.

clip_image003

In any case you can now administer VEEAM Backup & Replication without having to remote in to the server over RDP. The console will work over the LAN, WAN, or a VPN. Just make sure you have about 1Mbps bandwidth available to get the job done. Less than that and you might not find the experience very good. I suggest you test this to see how this works for you as your mileage may vary.

No more RDP ever?

Will I throw away my remote and secured RDP Gateway setup now I have a console? No. For one not every environment will let me connect over VPN. A locked down well secured RDS Gateway setup can provide for very save remote access with basically just keyboard, video, mouse and sound. Add two factor authentication for a more secure solution.. The ability to block the mapping of drives, printers, clipboard etc. secures against dropping content or files form a remote machine into your business environment.

Also, RDP has UDP available since Windows Server 2012 and that is an exceptionally marvelous tools to have when connecting over bad, low quality connections. It is amazing how good it works under such conditions. Even if do not want to RDP to the VEEAM Backup & Replication server I could RDP into a remote management workstation or server and use the console from there to connect to the Veeam Backup & Replication v9 server(s).

Maximum bandwidth in Hyper-V storage QoS policies

Introduction

In a previous blog post Hyper-V Storage QoS in Windows Server 2016 Works on SOFS and on LUNs/CSV I have discussed Storage QoS Policies in Windows Server 2016. I have also demonstrated this in a lab setup at VEEAMON 2015 in one of my talks at the Microsoft presentation area. It’s one of those features where a home lab will do the job. There is no need for special storage hardware. It’s all in box functionality. Cool!

Maximum bandwidth in Hyper-V storage QoS policies

Now that was in the Technical Preview 2 and 3 era, where it all revolved around minimum and maximum QoS. In Windows Server 2016 Technical Preview 4 we got some new features in regards to storage QoS policies. One of those is that we can now also set the Maximum bandwidth on a policy using the parameter MaximumIOBandwidth. This parameter, which is set in bytes per second determines the maximum bandwidth that any flow assigned to the policy is allowed to consume.

image

We use that policy ID to assign it to the 2 shared virtual disks of our cluster nodes. You’ll need to do this for all of the guest cluster nodes.image

You can copy the PoSh demo script below

[sourcecode language=”powershell”]

#Create a Storage Policies
$DemoVMPolicy = New-StorageQosPolicy -Name DemoVMPolicy -PolicyType MultiInstance `
-MinimumIops 250 -MaximumIops 500 -MaximumIOBandwidth 100MB

#Look at our storage Policies
Get-StorageQosPolicy -name DemoVMPolicy

#Grab our policy ID
$DemoVMPolicy = (get-StorageQosPolicy -Name DemoVMPolicy).PolicyId
$DemoVMPolicy

#Look at our VMs policy setting before and after assigning a storage policy.
#We assign the storage policy to the 2 shared virtual disks
#that are located a location 1 and 2 on SCSI controller 0

Get-VM -Name GuestClusterNode1 | Get-VMHardDiskDrive |
ft Path,MinimumIOPS, MaximumIOPS, MaximumIOBandwidth, QoSPolicyID -AutoSize

Get-VM -Name GuestClusterNode1 | Get-VMHardDiskDrive | Where-Object {$_.controllerlocation -ge 1}|
Set-VMHardDiskDrive -QoSPolicyID $DemoVMPolicy

Get-VM -Name GuestClusterNode1 | Get-VMHardDiskDrive |
ft Path, MinimumIOPS, MaximumIOPS, MaximumIOBandwidth, QoSPolicyID -AutoSize
[/sourcecode]

You can use MaximumIOBandwidth by itself or you can combine it with the maximum IOPS setting. When both of these parameter are set in a storage QoS policy they are both active. The one that is reached first by a flow assigned to this policy will be the limiting factor in the I/O of that flow.

As an example. Let’s say you specify 500 IOPS and 100Mbps bandwidth as maxima. Your workload hits 500 IOPS but only consumes 58 Mbps it’s the IOPS that are limiting the flow.