SFP+ and SFP28 compatibility

Introduction

As 25Gbps (SFP28) is on route to displace 10Gbps (SFP+) from its leading role as the work horse in the datacenter. That means that 10Gbps is slowly but surely becoming “the LOM option”. So it will be passing on to the role and place 1Gbps has held for many years. What extension slots are concerned we see 25Gbps cards rise tremendously in popularity. The same is happening on the switches where 25-100Gbps ports are readily available. As this transition takes place and we start working on acquiring 25Gbps or faster gear the question about SFP+ and SFP28 compatibility arises for anyone who’s involved in planning this.

SPF+ and SFP28 compatibility

Who needs 25Gbps?

When I got really deep into 10Gbps about 7 years ago I was considered a bit crazy and accused of over delivering. That was until they saw the speed of a live migration. From Windows Server 2012 and later versions that was driven home even more with shared nothing and storage live migration and SMB 3 Multichannel SMB Direct.

On top of that storage spaces and SOFS came onto the storage scene in the Microsoft Windows server ecosystem. This lead us to S2D and storage replica in Windows Server 2016 and later. This meant that the need for more bandwidth, higher throughput and low latency was ever more obvious and clear. Microsoft has a rather extensive collection of features & capabilities that leverage SMB 3 and as such can leverage RDMA.

In this time frame we also saw the strong rise of All Flash Array solutions with SSD and NVMe. Today we even see storage class memory come into the picture. All this means even bigger needs for high throughput at low latency, so the trend for ever faster Ethernet is not over yet.

What does this mean?

That means that 10Gbps is slowly but surely becoming the LOM option and is passing on to the role 1Gbps has held for many years. In our extension slots we see 25-100Gbps cards rise in popularity. The same is happening on the switches where we see 25, 50, 100Gbps or even higher. I’m not sure if 50Gbps is ever going to be as popular but 25Gbps is for sure. In any case I am not crazy but I do know how to avoid tech debt and get as much long term use out of hardware as possible.

When it comes to the optic components SFP+ is commonly used for 10Gbps. This provides a path to 40Gbps and 100Gbps via QSFP. For 25Gbps we have SFP28 (1 channel or lane for 25Gbps). This give us a path to 50Gbps (2225Gbps – two lanes) and to 100Gbps (4*25Gbps – 4 lanes) via QSFP28. In the end this a lot more economical. But let’s look at SFP+ and SFP28 compatibility now.

SFP+ and SFP28 compatibility

When it comes to SFP+ and SFP28 compatibility we’re golden. SFP+ and SFP28 share the same form factor & are “compatible”. The moment I learned that SFP28 share the same form factor with SFP+ I was hopeful that they would only differ in speed. And indeed, that hope became a sigh of relief when I read and experimentally demonstrated to myself the following things I had read:

  1. I can plug in a SFP28 module into an SFP+ port
  2. I can plug in a SFP+ module into an SFP28 port
  3. Connectivity is established at the lowest common denominator, which is 10Gbps
  4. The connectivity is functional but you don’t gain the benefits SFP28 bring to the table.

Compatibility for migrations & future proofing

For a migration path that is phased over time this is great news as you don’t need to have everything in place right away from day one. I can order 25Gbps NIC in my servers now, knowing that they will work with my existing 10Gbps network. They’ll be ready to roll when I get my switches replaced 6 months or a year later. Older servers with 10Gbps SFP+ that are still in production when the new network gear arrives can keep working on new SFP28 network gear.

  • SFP+: 10Gbps
  • SFP28: 25Gbps but it can go up to 28 so the name is SFP28, not 25. Note that SFP28 can handle 25Gbps, 10Gbps and even 1Gbps.
  • QSFP28: 100Gbps to 4*25Gbps or 2*50Gbps gives you flexibility and port density.
  • 25Gbps / SFP28 is the new workhorse to deliver more bandwidth, better error control, less cross talk and an economical sound upgrade path.

Do note that SFP+ modules will work in SFP28 ports and vice versa but you have to be a bit careful:

  • Fix the ports speed when you’re not running at the default speed
  • On SFP28 modules you might need to disable options such as forward error correction.
  • Make sure a 10Gbps switch is OK with a 25Gbps cables, it might not.

If you have all your gear from a vendor specializing in RDMA technology like Mellanox this detects this all this and takes care of everything for you. Between vendors and 3rd party cables pay extra attention to verifying all will be well.

SFP+ and SFP28 compatibility is also important for future proofing upgrade paths. When you buy and introduce new network gear it is nice to know what will work with what you already have and what will work with what you might or will have in the future. Some people will get all new network switches in at once while others might have to wait for a while before new servers with SFP28 arrive. Older servers might be around and will not force you to keep older switches around just for them.

SFP28 / QSFP28 provides flexibility

Compatibility is also important for purchase decision as you don’t need to match 25Gbps NIC ports to 25Gbps switch ports. You can use the QSFP28 cables and split them to 4 * 25Gbps SFP28.

SPF+ and SFP28 compatibility

QSFP28

The same goes for 50Gbps, which is 100Gbps QSFP to 2 * 50Gbps QSFP.

SPF+ and SFP28 compatibility

SPF+ and SFP28 compatibility

This means you can have switch port density and future proofing if you so desire. Some vendors offer modular switches where you can mix port types (Dell EMC Networking S6100-ON)

Conclusion

More bandwidth at less cost is a no brainer. It also makes your bean counters happy as this is achieved with less switches and cables. That also translates to less space in a datacenter, less consumption of power and less cooling. And the less material you have the less it cost in operational expenses (management and maintenance). This is only offset partially by our ever-growing need for more bandwidth. As converged networking matures and becomes better that also helps with the cost. Even where economies of scale don’t matter that much. The transition to 25Gbps and higher is facilitated by SFP+ and SFP28 compatibility and that is good news for all involved.

Storage Migration Service or MSFT adds WorkingHardInIT like skills in Windows Server 2019

As readers of my blog will know very well is that one of my track records is that I’m fairly successful in keeping technology debt low. One of the workloads I have always moved successfully over the years are file servers. Right now, almost all, almost everywhere, are the goodness of SMB3 and are high available clustered file server roles. You can read about some of those efforts in some of my blogs:

Sometimes this cans quite a challenge but over the years I have gained the experience and expertise to deal with such projects. Not everyone is in that position. This leads to aging file services, technology debt, security risks, missed opportunities (SMB3 people!) and often even unsupported situations in regards to hardware and software. While since the release of Windows Server 2016 this image has shifted a bit, you can clearly see the pretty depressing state of file services. Windows rules the on premises server world but the OS versions are aging fast.

clip_image002

Image courtesy of SpiceWorks: Server Virtualization and OS Trends

The operating systems are ancient, old or aging and we all know what this means in regards to the SMB version in use.

Now I work hard, effective and efficiently but I cannot be everywhere. Luckily for you Microsoft has a great new capability coming up in Windows Server 2019, the Windows Server Storage Migration Service.

clip_image004

Figure2: the WorkingHardInIT feature officially known as the Windows Server Storage Migration Service. Image courtesy of Microsoft.

Thanks to Ned Pyle and his team’s efforts you can no aspire to me as successful as me when it comes to migrating file services to newer environments. It’s like having WorkingHardInIT in a Windows feature. Isn’t that cool?! If they sold that separately they would make a pretty penny but they luckily for you include it with their new server OS version.

Storage Migration Servers deals with many of the usual problems and intricacies of a storage / file service migration. It will handle things like share settings, security settings, network addresses and names, local accounts, encrypted data, files that are in use etc. To handle you project you have a GUI and PowerShell automation at your disposal. Windows Server 2019 is still being perfected and you can still provide feedback while testing this feature.

Things to note for now (bar the requirements for testing as described in Ned Pyle’s blog) are:

Supported source operating systems VM or hardware (to migrate from):

  • Windows Server 2003
  • Windows Server 2008
  • Windows Server 2008 R2
  • Windows Server 2012
  • Windows Server 2012 R2
  • Windows Server 2016
  • Windows Server 2019 Preview

Supported destination operating system VM or hardware (to migrate to):

  • Windows Server 2019 Preview*

* Technically your destination for migration can be any Windows Server OS, but we aren’t testing that currently. And when we release the faster and more efficient target proxy system in a few weeks it will only run on Windows Server 2019.

In my humble opinion that almost has to be Storage Replica technology being leveraged. Something that has been proven to be very much more efficient that copying files. Microsoft already promotes Storage Replica to other server or itself as a way of moving data to a new LUN.

Anyway, this is a cool feature that should grab your attention. Thank you Ned Pyle and team! And while you’re busy putting this great capability into Windows Server 2019 (Standard and Datacenter) consider doing the same for full featured Storage Replica in Windows Server 2019 Standard .

Storage Replica Standard

Introduction

You probably have read by now that is  Storage Replica will be available on Windows Server 2019 Standard Edition. Until now with Windows Server 2016 and 2019 Previews it was a Datacenter Edition only feature. If you haven’t go read up on it over at

Announcing Windows Server 2019 Insider Preview Build 17639. This is good news and I’m very curious to see what use cases and solutions people will leverage this capability for.

Storage Replica on Standard Edition has 2 major limitations:

  • SR replicates a single volume (instead of an unlimited number of volumes).
  • Volumes can have one partnership (instead of an unlimited number of partners). Volumes can have a size of up to 2 TB (instead of an unlimited size).

My first use case

This is why I’m a bit disappointed right now as I had hoped it would help me with some data protection use cases. As part of my roles and expertise lies in building creative, cost effective data protection solutions adhering to at least the 3-2-1 rule in both on premises as well as in private, hybrid and public cloud setting or any combination of the above.

On of the challenges is getting backup data off site. One solution is to replicate the backups. This mostly doesn’t require synchronous replication as most of the time the bandwidth isn’t there or there is no need. Many cost effective storage solutions don’t have storage replication or it comes a significant license cost. The backup software often has this but it works a per definition less effective layer than Storage Replication.

Cost effective but effective backup targets don’t always need to be high available, that depends on the needs and you can cluster a backup target file server role with Windows Server standard edition.

So for my use case I would rather have seen asynchronous only in Windows Server 2019 Standard Edition but without the volume size and number limitation and be done with it.

This would give me off site data protection of my backups via storage Replica which is tremendously efficient.  It doesn’t rely on backup software or other solution operating at the file level and as such that are inherently less efficient.  But backups often mean more then one target volume and larger volume sizes than 2TB. The beauty of Storage Replica is that it’s completely storage agnostic solution and we can build the solution on top of whatever is at hand.

Some data protection use cases where the limits might not matter

With the current limits it might fit in with some SOHO/ ROBO scenario’s perhaps. When you have 10 branch offices with Standard Edition you could potentially replicate a volume to a central datacenter edition for safe keeping off site. But in those scenarios we’re also looking at Azure File Sync and offload the data protection to the cloud if/when possible.

Also remember that “pure” MPLS isn’t the only answer anymore to many connectivity challenges anymore but SDWAN/local Internet breakouts are eating part of their cake . MPLS costs versus VPN is also a reality which limits bandwidth (let’s face it, it’s a cost issue), perhaps further reducing desirability Storage Replica for this use case. SD-WAN could be helping address “old school” network cost optimization limitations for this use case by delivering more and better than VPN without the need for MPLS or express route etc. Just thinking out loud here. It’s a cloud first world, where servers still have a role to play, but for that they need to be flexible and allow for many possible permutations. It’s a missed opportunity I’d say. My opinion, but there is hope. Microsoft states the following on Storage Replica Standard

We will continue to listen to your feedback and evaluate these settings through our telemetry during Insider previews of Windows Server 2019. These limitations may change several times during the preview phase and at RTM.

So if my use case make enough sense to them they might change something still but we’ll  see. Anyway, my 5$ cents on the subject.