ReFS vNext Block Cloning and ODX

Introduction

With Windows Server 2016 we also get a new version of ReFS, which I’ll designate aptly as ReFS vNext. It offers a few new abstractions that allow applications and virtualization to control how files are created, copied and moved. The features that are crucial to this are block cloning and data tiering.

Block cloning allows to clone any block of a file into any other block of another file. These operations are based on low cost metadata actions. Block cloning remaps logical clusters (physical locations on a volume) form the source region to the destination region. It’s important to note that this works within the same file or between files. This combined with “allocate on write” ensures isolation between those regions, which basically means files will not over write each other’s data if they happen to reference the same region and one of them writes to that region. Likewise, for a single file, if a region is written to, that changed data will not pop up in the other region. You can learn more about it on this MSDN page on block cloning which explains this further.

ReFS vNext does not do this for every workload by default. It’s done on behalf of an application that calls block cloning, such as Hyper-V for example when merging VHDX files. For these purposes the meta data operations counting references to regions make data copies within a volume fast as it avoids reading and writing of all the data when creating a new file from an existing one, which would mean a full data copy. It also allows reordering data in a new file as with checkpoint merging and it also allows for “data projection” where data can be projected form one area in to another without an actual copy. That’s how speed is achieved.

Now some of the benefits of ReFS vNext are tied into Storage Space Direct. Such as the tiering capability in relation to the use of erasure encoding / parity to get the best out of a fast tier and a slower tier without losing too much capacity due to multiple full data copies. See Storage Spaces Direct in Technical Preview 4 for more information on this.

I’m still very much a student of all this and I advise you to follow up via blogs and documentation form Microsoft as they become available.

What does it mean?

In the end it’s all about making the best use of available resources. The one that you already have and the one that you will own in the future. This lowers TCO and increases ROI. It’s not just about being fast but also optimizing the use of capacity while protecting data. There is one golden rule in storage: “Thou shalt not lose data”.

For now, even when you’re not yet in a position to evaluate Storage Space Direct, ReFS vNext on existing storage show a lot of promise as well. I have blogged about file creation speeds (VHDX files) in this blog post: Lightning Fast Fixed VHDX File Creation Speed With ReFS vNext on Windows Server 2016. In another blog post, Accelerated Checkpoint merging with ReFS vNext in Windows Server 2016 you can read about the early results I’ve seen with Hyper-V checkpoint merging in Windows Server 2016 Technical Preview 4. These two examples are pretty amazing and those results are driven by ReFS metadata operations as well.

Does it replace ODX?

While the results so far are impressive and I’m looking forward to more of this, it does not replace ODX. It complements it. But why would we want that you might ask, as we’ve seen in some early testing that it seems to beat ODX in speed? It’s high time to take a look at ReFS vNext Block Cloning and ODX in Windows Server 2016 TPv4.

The reality is that sometimes you’ll probably don’t want ODX to be used as the capabilities of ReFS vNext will provide for better (faster) results. But sometimes ReFS vNext cannot do this. When? Block cloning for all practical purposes works within a volume. That means can only do its magic with data living on the same volume. So when you copy data between two volumes on the same LUN or between volumes on a different LUNs you will not see those speed improvements. So for deploying templates stored on another LUN/CSV fast it’s not that useful. Likewise, if for space issues or performance issues you were storing your checkpoints on a different LUN you will not see the benefits of ReFS vNext block cloning when merging those checkpoints. So you will have to revise certain design and deployment decisions you made in the past. Sometimes you can do this, sometime you can’t. But as ODX works at the array level (or beyond in certain federated systems) you can get excellent speeds wile copying data between volumes / LUNs on the same server, between volumes / LUNs on different servers. You can also leverage SMB 3.0 to have ODX kick in when it makes sense to avoid senseless data copies etc. So ODX has its own strengths and benefits ReFS vNext cannot touch and vice versa. But they complement each other beautifully.

So as ReFS vNext demonstrates ODX like behavior, often outperforming ODX, you cannot just compare those two head on. They have their own strengths. Just remember and realize that ReFS vNext actually does support ODX so when it’s applicable it can be leveraged. That’s one thing I did not understand form the start. This is beginning to sound like an ideal world where ReFS vNext shines whenever its features are the better choice while it can leverage the strengths of ODX – if the underlying storage array provides it – for those scenarios where ReFS vNext cannot do its magic as described above.

The Future

I’m not the architect at Microsoft working on ReFS vNext. I do know however, that a bunch of very smart people is working on that file system. They see, hear and listen to our experiments, results, and requests. ReFS is getting a lot of renewed attention in Windows Server 2016 as the preferred file system for Storage Space Direct and as such for CSVs. Hyper-V is clearly very much on board with leveraging the capabilities of ReFS vNext. The excellent results of that, which we can see in speeding up VHDX creation/ extending and checkpoint merges, are testimony to this. So I’m guessing this file system is far from done and is going places. I’m expecting more and more workloads to start leveraging the ReFS vNext capabilities. I can see ReFS itself also become more and more feature complete and for example Microsoft has now stated that they are working on deduplication for ReFS, although they do not yet have any specifics on release plans. It makes sense that they are doing this. To me, a more feature complete ReFS being leveraged in ever more uses cases is the way forward. For now, we’ll have to wait and see what the future holds but I am positive albeit a bit impatient. As always I’m providing Microsoft with my feedback and wishes. If and when they make sense and are feasible they probably have them on their roadmap or I might give them an idea for a better product, which is good for me as a customer or partner.

Happy New Year from a renewed Microsoft MVP in 2016

Happy New Year from a renewed Microsoft MVP in 2016

It’s January 1st 2016, late in the afternoon here local time and I have just received great news to start the new year with. It came by way of an e-mail notifying me I have been renewed as a Microsoft Most Valued Professional (MPV).

The Microsoft MVP Award provides us the unique opportunity to celebrate and honor your significant contributions and say “Thank you for your technical leadership.”

image

So it’s time for a happy New year from a renewed Microsoft MVP in 2016. My expertise is now Cloud and Data Center Management. It’s quite an honor to be renewed. Somewhere people think I make a big enough difference to be recognized, that caresses my ego just a little bit. More importantly however it means I get the opportunity to keep working with a lot of passionate and talented people. The ability to participate in a global community and ecosystem focused on our areas of expertise is something I have enjoyed for many years now. Attending the MVP Summit is the cherry on the cake and they sure do make you feel welcome at every place you stop on and around the campus.

clip_image001[5]

My fellow MVPs are always very helpful, they are both an inspiration as well as a source of tremendous experience and knowledge. Being a MVP has opened opportunities to both learn and teach, both professionally and personally. That’s what enabled me to grow in depth and breadth within my areas of expertise which ultimately translates into our new expertise assignment, cloud and datacenter management.

Thank you!

It’s a good time to wish you all a happy New Year. Let me take a moment to express my gratitude to all loyal or accidental readers of WorkingHardInIT. A blog without readers would be a sad thing but luckily you’re all reading this blog more and more, year after year.

image

I’m grateful you for your continued support and spending the time reading my blog. To the people, businesses and organizations that given me so many opportunities and support and with whom I had the pleasure to work with in 2015, I say thank you and let’s continue to do so. I wish you all a marvelous 2016 with lots of joy, good health and tons fun at and outside of work!

The road ahead

2016 will be an interesting year. There’s a lot going on in our industry, some of it is hype, a lot of it is real. That reality is sometimes sobering but often inspiring. Keep cool, don’t panic or go ballistic. Smart discipline with a good portion of common sense, insights and a solid, yet flexible plan wins the day. You’ll also need some luck and turn up at the right place at the right time every now and then, ready to make the most of an opportunity. You get the idea.

There are and have been, as always, personal and professional challenges. That’s a given. Only newbies and idiots make picture perfect plans. They then get “dazzled” by the first punch on their snout which sends their plans falling apart like shattered glass. Sometimes the challenges are bigger and harder. This can mean you need to work even harder, smarter and perhaps even longer. It can also mean to cut your losses and disengage. No matter how good you are, how long, hard and smart you work, you cannot right all wrongs in this world. Leave that to the self-promoting LinkedIn blogs on “personal success and growth” aimed at ridiculously entitled people or the painfully naïve.

Importance

2016 will also know its challenges. They will be met with all the attention and dedication required where and when needed. They will be passed by or ignore where the effort just isn’t worthwhile. There’re good places to go, nice things to do and great people to meet. If I can seize as many opportunities in 2016 (TechEd, ITPROCeed, E2EVC, VEEAMON, Microsoft MVP Summit, ExpertsLive)  like I have been able to do in 2015 I’ll be a happy man, both professionally and personally.

How to get a dream job in 2016?

I’ve been asked that a couple of times. I’m not the one  for handing out personal advice, that would only shock your parents and potentially shake your worldview as well. Professionally I’d say, your profession, your career is not the same as your job. It might be, but more often than not it isn’t. That’s OK. You can build a career in your (chosen) profession even despite your job or jobs. Most MVPs work very hard and we put a lot of personal time into our technical skills and community. It isn’t a lifestyle of the rich and famous as some would think when you read a blog about a conference or summit.

VEEAMON 2015 Party

Those are a fun part of work, that’s for sure, but they don’t define our work days. It’s lots of work, learning, sharing and many battles are uphill!. We all have jobs that require us to do things we’d rather not have to do. Do what you need to do to stay afloat but try to do as much of what you like and enjoy it as possible. Do it smart and don’t waste your time or let others waste yours. The latter is something you should not do to other people either. When it comes to jobs it’s not all that simple as the sloganesque “Do what you love, versus work for money/the man/a pension/security” for most people. Sure most don’t like to admit that they have to take crap, but we all do. Anything else is as much BS as every employer that seems to pretend everybody has to be and is an engaged, inspired team player who’s going all out for the company, beyond and above what the job demands. That’s a bit too much like Office Space’s “Is this good for the company?” for comfort 😉

Microsoft MVP Global Summit 2015

We’re leaving Las Vegas (VEEAMON 2015) and are heading for Seattle.

image

I’ll be spending a week in Bellevue/Redmond from November 1st to November 8th to attend the Microsoft MVP Global Summit 2015. As a Microsoft MVP one does not want to miss this summit. And I know quite a few MVPs who have quit their jobs the moment their boss told ‘m they could not go. Not saying we’d all do that but it’s true that skilled experts are in high demand and missing out on the MVP summit isn’t exactly making the best use of any MVP you employ.

It’s all NDA so basically I can’t tell you anything. But we’ll be sharing some tweets if joy and some lightweight blogs to share with the world how happy we MVPs are to be back at the mother ship for our annual firmware downloads and scheduled maintenance.

Until then here’s a nice Bing Maps overview of the MVP before & after hours area of operation.image

So yes, the Microsoft MVPs are descending on Bellevue / Redmond once again in large numbers. So you might hear a bit more Microsoft technology discussions in a bit more foreign languages or accents than otherwise. It’s us, no worries!

image

It’s just great to an MVP Smile

Hit me baby one more time or the Faster Fast Ring of Windows 10 Insider Builds

Hit me baby one more time!

This blog is brought to you by Francesco V. Buccoli, a brilliant ex Hyper-V MVP who went blue badge and became a PFE. Why? Because he called me a genius, that’s why!

image

Here we go again, things are heating up in the last straight track towards RTM of Windows 10. We’re now getting build 10162 right on the heals of build 10159 that basically overran people who were still downloading 10158.

No this is not some PM in Redmond hitting the publish button by mistake again a la “Oops, I did it again” but it’s with intent and purpose. Deliver an awesome client right from the start.

So far it’s all good. The quality of these lasted builds, even during the limited time we get to spend with them, is very good and show real improvements over the entire line. Windows 10 should be ready for rollout at RTM/GA if the quality is this good and only improves.

We lead, we weren’t born to follow.

Now, go download it already and I’ll quit the cheesy music references Winking smile