Plug ‘n Play enumeration of devices has been very useful for loading device drivers automatically but isn’t deterministic. As devices are enumerated in the order they are received it will be different from server to server but also within the system. Meaning that enumeration and order of the NIC ports in the operating system may vary and “Local Area Connection 2” doesn’t always map to port 2 on the on board NIC. It’s random. This means that scripting is “rather hard” and even finding out what NIC matches what port is a game of unplugging cables.
Consistent Device Naming is the solution
A mechanism that has to be supported by the BIOS was devised to deal with this and enable consistent naming of the NIC port numbering on the chassis and in the operating system.
But it’s even better. This doesn’t just work with on board NICs. It also works with add on cards as you can see. In the name column it identifies the slot in which the card sits and numbers the ports consistently.
In the DELL 12th Generation PowerEdge Servers this feature is enabled by default. It is not in HP servers for some reason, you need to turn in it on manually.
I first heard about this feature even before Windows Server 2012 Beta was released but as it turns out Dell has been involved with the development of this feature. It was Dell BIOS team members that developed the solution to consistently name network ports and had it standardized via PCI SIG. They also collaborated with Microsoft to ensure that Windows Server 2012 would support all this.
Here’s a screen shot of a DELL R720 (12th Generation PowerEdge Server) of ours. As you can see the Consistent Device Naming doesn’t only work for the on broad NIC card. It also does a fine job with add on cards of which we have quite a few in this server.
It clearly shows the support for Consistent Device Naming for the add on cards present in this server. This is a test server of ours (until we have to take it into production) and it has a quad 1Gbps Intel card, a dual Intel X520 DA card and a dual port Mellanox 10Gbps RoCE card. We use it to test out our assumptions & ideas. We still need a Chelsio iWarp card for more testing mind you
A closer look
This solution is illustrated the in the “Device Name column” in the screen shot below. It’s clear that the PnP enumerated name (the friendly name via the driver INF file) and the enumerated number value are very different from the number in Name column ( NIC1, NIC2, NIC2, NIC4) even if in this case where by change the order is correct. If the operating system is reinstalled, or drivers changed and the devices re-enumerated, these numbers may change as they did with previous operating systems.
The “Name” column is where the Consistent Device Naming magic comes to live. As you can see you are able to easily identify port names as they are numbered consistently, regardless of the “Device Name” column numbering and in accordance with the numbering on the chassis or add on card. This column name will NEVER differ between identical servers of after reinstalling a server because it is not dependent on PnP. Pretty cool isn’t it! Also note that we can rename the Name column and if we choose we can keep the original name in that one to preserve the mapping to the physical hardware location.
In the example below thing map perfectly between the Name column and the Device Name column but that’s pure luck.
This feature helps, but isn’t a substitute manual checking. We had a series of R710s with Intel X520-T2 NICs. Four of the five cards were consistent, but one of the four cards presented a reverse device ID! So, even though its a step in the right direction, double check. 🙂
I’ve seen it also I think, but it’s been a while. Just out of interest, Dell provides “firmware” upgrades for the X520 (now at 15.0.28, http://www.dell.com/support/drivers/us/en/19/driverdetails?driverid=HKK1W ), do you have that deployed?
I’ll check. That had occurred to me; I just did the v502 set of updates, so I probably should have checked that. But, then I wonder if it will reverse them again! ;p
Didn’t help. Weird. As far as I can tell, the rest of the machine matches the other four in the cluster. Now, if I can get the Broadcom NICs to not start enumerating at #37… 🙂
Don’t get stuck focussing on those numbers, they’re all over the place depending on history of server. The important part is consistent. Port 1 as marked on HW is port 1 in Windows.
Well, its certainly not perfect, but a WHOLE lot better than it used to be. If this was the biggest problem I had in my lab, I’d be happy camper! Regardless, the device consistent naming is something I’ll call a good “starting point” but not reliable enough (in my experience) to depend upon (on the R710s anyway, maybe it will be better on the R820s). I’ll be doing manual verifications (if not checks) for a while still. 🙂
Regardless, it’s great to find someone else to chat about this sort of thing with!
PS Thanks for the informative blog!