Veeam Leads the way by leveraging ReFS v3 capabilities

Introduction

You might have noticed that I’m pretty impressed by what Microsoft is doing with ReFS v3 in Windows Server 2016. You can read some of my musing on it in ReFS vNext Block Cloning and ODX and take a look at a comparison between ReFS & ODX speeds when creating VHDX files in Lightning Fast Fixed VHDX File Creation Speed With ReFS on Windows Server 2016 .

Note that this is also leveraged for accelerated checkpoint merges, VHDX resizing etc.

Now it goes without saying that Hyper-V (they’re the tip of the spear at MSFT) and other Microsoft products would take advantage of the capabilities of ReFS. But now we know that Veeam Backup & Replication 9.5 has made use of ReFS to help with the resilience of their backups, the speed of their Synthetic Full backups and the space required.

image

To a Hyper-V MVP and a Veeam vanguard it was obvious these two combined just had to lead to way for others to follow.

Veeam Leads the way by leveraging ReFS v3 capabilities

Veeam Backup & Replication 9.5 will leverage ReFS v3 …

image

 

and by doing so they deliver the following benefits:

  • Shorted backup windows and a reduced backup storage load on the repository
  • Reduced backup target storage capacity which is reducing or eliminating the need for deduplication in many scenarios.
  • Better backup data protection by leveraging the ReFS native capabilities to protect against bit rot which was one of the prime goals for which Microsoft designed ReFS.

How is this done?

ReFS v3 has “fast cloning” technology which Veeam is leveraging. This results in up to 10 times  faster creation and transformation of synthetic full backup files!  ReFS fast cloning allows for creating new files without physically moving data blocks between files. This is what delivers even shorter backup windows and lower backup storage load on the repository or repositories.

They use what they call “Spaceless full backup technology” which allows multiple full backup files to reside on the same ReFS volume that share the same physical data blocks. As a result they need less storage capacity which can reduce or eliminate the need (and cost) of deduplication appliances whilst leveraging commodity storage.

Lest see how this is done. A “legacy” full backup is created an consumes 30% storage capacity. Then we make incremental backups.

image

3 incremental backups add 3 * 10% of delta to the needed backup storage capacity which adds up to 60%.

image

We create a synthetic full backup and the copies of the data require another 30% of space (90%). 

image

No let’s compare this to v9.5 that leverages a Windows Server 2016 ReFS formatted backup target repository. Instead of copying data ReFS references already existing data block for a new file. This saves on IO, space and time!

image

Is this safe? What if those data blocks that are reference multiple times are corrupted? Well Veeam does have protection against that in place already! But it goes the extra mile as ReFS has the capabilities to protect against that itself or it’s power would also become its biggest weakness.

Veeam’s data integrity streams integration leverages ReFS data integrity scanner and even proactive error correction when used in combination with Storage Spaces to protect backup files from bit rot and allows for more reliable forever-incremental archiving. This helps make the spaceless full backup technology trustworthy & safe alongside the health checking & error fixing capabilities already available in Veeam Backup & Replication.

Conclusion

I’m impressed by the forward looking and fast adoption of the capabilities of ReFS v3 by Veeam and I’m testing Backup & Replication v9.5 Beta today in the lab. They have more up their sleeve by the way as they have some interesting work with PowerShell Direct to make backups ever more resilient in ever more scenarios. More on that later.

Anyone who said Veeam would lose its edge in the world of Hyper-V backups when Microsoft introduced their own native change block tracking (resilient change tracking) has clearly never dealt with Veeam seriously and professionally. I have and I’m always happy to chat to them as they have serious technical skills combined with vision and business acumen that makes sure they’re leaders in the business of backup. It makes me proud to be a Veeam vanguard and a MVP with a specialization in Hyper-V.

ReFS vNext Block Cloning and ODX

Introduction

With Windows Server 2016 we also get a new version of ReFS, which I’ll designate aptly as ReFS vNext. It offers a few new abstractions that allow applications and virtualization to control how files are created, copied and moved. The features that are crucial to this are block cloning and data tiering.

Block cloning allows to clone any block of a file into any other block of another file. These operations are based on low cost metadata actions. Block cloning remaps logical clusters (physical locations on a volume) form the source region to the destination region. It’s important to note that this works within the same file or between files. This combined with “allocate on write” ensures isolation between those regions, which basically means files will not over write each other’s data if they happen to reference the same region and one of them writes to that region. Likewise, for a single file, if a region is written to, that changed data will not pop up in the other region. You can learn more about it on this MSDN page on block cloning which explains this further.

ReFS vNext does not do this for every workload by default. It’s done on behalf of an application that calls block cloning, such as Hyper-V for example when merging VHDX files. For these purposes the meta data operations counting references to regions make data copies within a volume fast as it avoids reading and writing of all the data when creating a new file from an existing one, which would mean a full data copy. It also allows reordering data in a new file as with checkpoint merging and it also allows for “data projection” where data can be projected form one area in to another without an actual copy. That’s how speed is achieved.

Now some of the benefits of ReFS vNext are tied into Storage Space Direct. Such as the tiering capability in relation to the use of erasure encoding / parity to get the best out of a fast tier and a slower tier without losing too much capacity due to multiple full data copies. See Storage Spaces Direct in Technical Preview 4 for more information on this.

I’m still very much a student of all this and I advise you to follow up via blogs and documentation form Microsoft as they become available.

What does it mean?

In the end it’s all about making the best use of available resources. The one that you already have and the one that you will own in the future. This lowers TCO and increases ROI. It’s not just about being fast but also optimizing the use of capacity while protecting data. There is one golden rule in storage: “Thou shalt not lose data”.

For now, even when you’re not yet in a position to evaluate Storage Space Direct, ReFS vNext on existing storage show a lot of promise as well. I have blogged about file creation speeds (VHDX files) in this blog post: Lightning Fast Fixed VHDX File Creation Speed With ReFS vNext on Windows Server 2016. In another blog post, Accelerated Checkpoint merging with ReFS vNext in Windows Server 2016 you can read about the early results I’ve seen with Hyper-V checkpoint merging in Windows Server 2016 Technical Preview 4. These two examples are pretty amazing and those results are driven by ReFS metadata operations as well.

Does it replace ODX?

While the results so far are impressive and I’m looking forward to more of this, it does not replace ODX. It complements it. But why would we want that you might ask, as we’ve seen in some early testing that it seems to beat ODX in speed? It’s high time to take a look at ReFS vNext Block Cloning and ODX in Windows Server 2016 TPv4.

The reality is that sometimes you’ll probably don’t want ODX to be used as the capabilities of ReFS vNext will provide for better (faster) results. But sometimes ReFS vNext cannot do this. When? Block cloning for all practical purposes works within a volume. That means can only do its magic with data living on the same volume. So when you copy data between two volumes on the same LUN or between volumes on a different LUNs you will not see those speed improvements. So for deploying templates stored on another LUN/CSV fast it’s not that useful. Likewise, if for space issues or performance issues you were storing your checkpoints on a different LUN you will not see the benefits of ReFS vNext block cloning when merging those checkpoints. So you will have to revise certain design and deployment decisions you made in the past. Sometimes you can do this, sometime you can’t. But as ODX works at the array level (or beyond in certain federated systems) you can get excellent speeds wile copying data between volumes / LUNs on the same server, between volumes / LUNs on different servers. You can also leverage SMB 3.0 to have ODX kick in when it makes sense to avoid senseless data copies etc. So ODX has its own strengths and benefits ReFS vNext cannot touch and vice versa. But they complement each other beautifully.

So as ReFS vNext demonstrates ODX like behavior, often outperforming ODX, you cannot just compare those two head on. They have their own strengths. Just remember and realize that ReFS vNext actually does support ODX so when it’s applicable it can be leveraged. That’s one thing I did not understand form the start. This is beginning to sound like an ideal world where ReFS vNext shines whenever its features are the better choice while it can leverage the strengths of ODX – if the underlying storage array provides it – for those scenarios where ReFS vNext cannot do its magic as described above.

The Future

I’m not the architect at Microsoft working on ReFS vNext. I do know however, that a bunch of very smart people is working on that file system. They see, hear and listen to our experiments, results, and requests. ReFS is getting a lot of renewed attention in Windows Server 2016 as the preferred file system for Storage Space Direct and as such for CSVs. Hyper-V is clearly very much on board with leveraging the capabilities of ReFS vNext. The excellent results of that, which we can see in speeding up VHDX creation/ extending and checkpoint merges, are testimony to this. So I’m guessing this file system is far from done and is going places. I’m expecting more and more workloads to start leveraging the ReFS vNext capabilities. I can see ReFS itself also become more and more feature complete and for example Microsoft has now stated that they are working on deduplication for ReFS, although they do not yet have any specifics on release plans. It makes sense that they are doing this. To me, a more feature complete ReFS being leveraged in ever more uses cases is the way forward. For now, we’ll have to wait and see what the future holds but I am positive albeit a bit impatient. As always I’m providing Microsoft with my feedback and wishes. If and when they make sense and are feasible they probably have them on their roadmap or I might give them an idea for a better product, which is good for me as a customer or partner.

Lightning Fast Fixed VHDX File Creation Speed With ReFS on Windows Server 2016

In this blog post we’re going to take a quick look at the lightning fast fixed VHDX file creation speed with ReFS on Windows Server 2016. We’ll compare it to creating fixed VHDX files On NTFS with a SAN that supports ODX. Both the NTFS and the CSV volume are CSV disk in a Hyper-V cluster and the test is run on the same node. The ODX cabale SAN is a Dell Compellent with Storage Center 6.5.20.

We create on  a selection of fixed VHDX files sizes (50GB, 100GB, 500GB and 1TB) on NTFS volume Windows Server 2016 host You can see the quite excellent results in file creation speeds with ODX.

image

These results are very good (DELL Compellent always did a great job implementing ODX) and the time to create a 1TB fixed VHDX is just over 5 seconds consistently. Impressive by any standard I would say! When we start using CSVs we can see that times double for the larger VHDX sizes but still +/- 12seconds for a 1TB disk is impressive by any standard. There is little difference whether the node where the script runs owns the CSV or not.

image

image

Can things be more impressive? Let’s do the same exercise on a ReFS volume on a Windows Server 2016 host. Same server, same SAN with ODX enabled but note that ReFS does not even support ODX, so it cannot be leveraged.

image

No matter what the file size of our fixed VHDX files they are created in just over 1 second consistently. This is very impressive.

imageimage

When we use a CVS LUN we still see the same impressive results. On CSV LUNS not owned by the node where we execute the test script we see a creation time of 2 seconds for VHDX sizes of 1TB. Still lightning fast.

If you do not have a SAN that supports ODX you can see why ReFS might become a very attractive choice for the file system for your Hyper-V virtual machine data volumes in Windows Server 2016. I can see why they mentioned it as the preferred option for Storage Spaces Direct. Do note that ReFS does not support deduplication and/or UNMAP (I see no dedupe support yet for virtual server workloads on the horizon either yet?). If you move large amounts of data around ODX does provide significant assistance with this. So with ReFS go for a large SSD tier. Flash only without deduplication or erasure coding might be cost prohibitive I’m afraid.

But let this not put you off ReFS. It has many benefits in combination with storage spaces and these new VHDX operation capabilities just add to that. So for many environments with commodity based storage this has become an even more interesting choice.

REFS Is Going Places At Microsoft Ignite 2015

I just loved the strengths of ReFS when I started looking into them when it was first announced. However it has been a bit quiet around ReFS due to some limitations or support issues.

But we’re seeing progress again and it seems that ReFS will be taken on a bigger role, even as the preferred file system for certain use cases. This is awesome. We’ll get the benefits REFS brings for less expensive types of disks in combination with storage spaces which are quite good already:

  • Integrity: ReFS stores data so that it is protected from many of the common errors that can cause data loss. File system metadata is always protected. Optionally, user data can be protected on a per-volume, per-directory, or per-file basis. If corruption occurs, ReFS can detect and, when configured with Storage Spaces, automatically correct the corruption. In the event of a system error, ReFS is designed to recover from that error rapidly, with no loss of user data.
  • Availability: ReFS is designed to prioritize the availability of data. With ReFS, if corruption occurs, and it cannot be repaired automatically, the online salvage process is localized to the area of corruption, requiring no volume down-time. In short, if corruption occurs, ReFS will stay online.
  • Scalability: ReFS is designed for the data set sizes of today and the data set sizes of tomorrow; it’s optimized for high scalability.
  • App Compatibility: To maximize AppCompat, ReFS supports a subset of NTFS features plus Win32 APIs that are widely adopted.
  • Proactive Error Identification: The integrity capabilities of ReFS are leveraged by a data integrity scanner (a “scrubber”) that periodically scans the volume, attempts to identify latent corruption, and then proactively triggers a repair of that corrupt data.

But, there’s more as ReFS has been improved and those improvements qualify it as the best default choice for a file system on Storage Spaces Direct (SSDi or D2S). What they have done to make it fast for certain data operations (VM creation, resizing, merges, snapshots) can only be described as “ODX” like. We’re getting speed, scalability, auto repair, high availability and data protection in a budget friendly storage. What’s not to like. There are many uses cases now where the need and benefits are clear but the economics worked against us. Well that’s about to be solved with the new and improved storage offerings in Windows Server 2016. I’m looking forward to more information on the evolution of ReFS as it matures as a file systems and take its place in on the front stage over the years. If you haven’t yet, I suggest you start looking at ReFS (again). I’ll be watching to see how far they take this.