Response and feedback about the Veeam hardened repository presentation

Veeam hardened repository presentation

It is great to get so much response and feedback about the Veeam hardened repository presentation. So, first of all, I thank all those reaching out to me in regards to the TechNine virtual user group session I gave on this subject.

Response and feedback about the Veeam hardened repository presentation
Immutability and backup chains – this was a well received presentation!

Most of you did so via the contact e-mail on my blog and not via a comment on the original blog post here. Many of you asked for a recording. Let’s address that first.

Where is the recording?

There is no recording. The TechNine user group tries to bring people together for the events and promote interactive discussions afterward. Hence, no recording is available. They do this to stimulate participation.

I did not make this presentation exclusively for TechNine, but they did get the world premiere. The good news is that I will be giving this session again and I will even be doing a webcast with my fellow Microsoft MVP and Veeam Vanguard Carsten Rachfahl about this. In the webcast, we will discuss the hardened repository at length and it will be published online.

So, nothing to worry about, you will get other opportunities to attend and you will have a recoding of the content reasonably soon.

Reaching out to me

Lately, I have noticed that my readers and social media followers seem to have gotten increasingly shy and do not ask their questions publicly via the blog comments section or social media.

That poses a challenge to me. While I would like to help you all individually, that approach just doesn’t scale. I have a job, family, life, and interest to pursue. I just cannot allocate the time to do so.

Veeam hardened repository presentation
I do explain this on my contact page!

Please ask you questions in the comments section of the relevant blog post and I will normally get to it. The benefit is that that public answer can help others as well with limited effort from myself. If the answer to a super interesting question is lengthy, I can decide to turn it into a blog post. That also helps all people out there. Thank you for being respectful of my time and sharing with the community yourself!

The Veeam hardened repository at the TechNine User group

The Veeam hardened repository

Just a quick blog post to let you all know I will be presenting on the Veeam hardened repository in Veeam Backup & Replication v11. I will do this at the TechNine user group meeting on March the 23rd. It is the world premiere of this presentation, so grab your seat and be able to tell your grandchildren you were there!

The Veeam hardened repository
The hardened repository with immutable backups

There is a tremendous amount of new and enhanced capabilities in v11, both small and big ones. But this one stands out because no matter who you are, a small business or a fortune 500 global enterprise, we are all at risk from ransomware and wipers. And guess what, the Veeam hardened repository gives you yet another option in your arsenal to make sure you can recover from such a disaster.

And end to end solution

In VBR v11 Veeam now offers complete end to end immutability for all your backup copies if you desire to do so. You can bring your own storage and so it leaves you the flexibility and freedom of choice you are used to with Veeam.

Like many of you, I like to stay up to date with new technology and implement it to solve real-world problems. Often coo tech is very forward-looking and foundational. This translates into “it is very early days yet”, “the code exists but you cannot use it yet”.

The Veeam hardened repository
You cannot delete immutable backups

Well, this is not something that shows you the promises of tomorrow and then leaves you to go back empty-handed. That always feels like a bit of a letdown when you are dealing with your real-world, real-time issues. The Veeam hardened repository is available now to help address the challenges you are facing today! It doesn’t get any better than that. I will show you the cookie, you can have the cookie and you can eat the cookie! Want to play with it yourself? You can, right now with the free community edition of Veeam Backup & Replication v11!

Register!

What do you need to do? Register of cause! Do it right here. Make sure you show up in time and stay around for an interactive discussion afterward. We start at 20:00 on March 23rd, 2021. And yes it is all virtual as we plan to get rid of the coronavirus first and then get together in real life again.

The Veeam hardened repository
register here

I will explain why you need it, how to set it up, and I will dive under the hood to show how it gets the job done. You can now protect your hypervisor and supported backups end to end with immutability in the hardened repository, giving you the extra security and protection you might need to recovery from a ransomware event. There will be demos! You can read up on it here but I will go behind the scenes with you!

I am a Veeam Vanguard 2021

I am a Veeam Vanguard 2021

This afternoon I received an email notifying me that my 2021 Veeam Vanguard Renewal Nomination has been approved! That means I am a Veeam Vanguard 2021!

I am a Veeam Vanguard 2021
Honored and proud to be a Veeam Vanguard again in 2021!

The Veeam Vanguard Program

You can read all about the Veeam Vanguard Program here. It is Veaam’s top-level influencer community. We share our insights, provide feedback, and help each other, as well as Veeam, succeed. The Veeam Vanguards are a varied group of people and combine a wide range of expertise in different verticals and in different disciplines. We all contribute to the community at large in different ways. What unites us is that we all share a passion for our industry and show technical thought leadership for the different technical communities in which Veeam exists.

Now I would not call myself a technical thought leader but I do know that via this program I get to stand on the shoulders of giants. Doing so gives me a majestic view over this industry segment, a view I would not have without it.

Thank you!

This is a very nice way to head into the weekend. I am both honored and proud to be a Veeam Vanguard. Veeam as a company has won my respect many years ago and it has only grown. They focus on customer value, deliver an excellent high-quality product, offer top-notch support, and finally offer great tools to the community. I can honestly say that I enjoy working with their products as well as the company and the individuals I know at Veeam.

I am looking forward to another year in this program and the opportunities this brings to us all for sharing, learning, and growing.

Thank you, we’ll make 2021 a good community year!

Immutability of Linux files on the Veeam hardened Linux repository

Introduction

Without any doubt, I find the hardened Linux repository Veeam introduces in Veeam Backup & Replication v11 one of the most fascinating new features to get my hands on. In the ever-escalating battle with ransomware and wipers, this is a very valuable option to have in your defensive arsenal. So, I grabbed the Beta 2 and got to work in the lab over the holidays to investigate and find out some details about the Immutability of Linux files on the Veeam hardened Linux repository.

Immutability of Linux files on the Veeam hardened Linux repository

I’s quite easy to find the file attribute “i” that marks a file as immutable.

attr -a -l attr -a -l 
Immutability of Linux files on the Veeam hardened Linux repository
lsattr -a also shows the hidden files

or

attr -a -l
Immutability of Linux files on the Veeam hardened Linux repository
lsattr -a -l list out the full name of the attribute.

Where is the information about the immutability actually stored? I mean, that “i” attribute is one thing but how do the Linux host and Veeam know from what time period this immutability is valid. In the end, the service has to clear it and know when to do this. Or is this only stored in the Veeam database or both?

How does it now from when till when a file must be immutable?

Digging around in the files and folders of the Veeam repository, I soon found the lock file “.veeam.x.lock” (see the green arrow in the image above) that is created by the veeamimmureposvc service. The owner is root, hence it is not created by the Veeam transport service. The veeamimmureposvc service is a local account with root access for managing the immutability. It only works locally and does not listen on any network port, hence it cannot be accessed remotely.

Immutability of Linux files on the Veeam hardened Linux repository
The veeamimmureposvc service controls the .veeam.x.lock file. the x is a number has increments with every backup job you run.

Let’s look inside to see if we can read something there?

cat .veeam.9.lock
the lock file is an XML file containing all the date/time stamps for every file in that backup job.

When you open that file you will find it to be an XML file. Inside you’ll see the date and time stamp for every file in the backup chains for that job. That’s cool.

But there is more. When we run to look for extend file attributes we find that every Veeam created file has a one called user.immutable.until.

The backup files all have an extended file attribute called user.immutable.until.

With that name, it is clear it can be of interest to us. If you look at what is in there, you’ll see it contains the date and time stamp for that file’s immutability period.

getfattr * -n user.immutable.unil 
Immutability of Linux files on the Veeam hardened Linux repository
The extended file attribute contains the timestamp until when that backup file is immutable!

That I find interesting. Veeam saves the information twice. Is that for redundancy or as some sort of checksum? Maybe it also has to do with the fact Veeam backup files are transportable and self-contained so that information is stored as an extended file attribute.

Conclusion

So there you have it. A small piece of information on where the immutability information is stored. The most surprising thing to me was that it is actual stored twice.

I hope you fund this interesting. Poking around to figure out the how and what of things always helps me tremendously to learn and understand the technologies I want to work with. That leads to better decisions in design and implementation. It leads to both trust and confidence, which helps me decide where and when to leverage it. Finally it also, almost without, it is invaluable when supporting the technology.