The Veeam hardened repository at the TechNine User group

The Veeam hardened repository

Just a quick blog post to let you all know I will be presenting on the Veeam hardened repository in Veeam Backup & Replication v11. I will do this at the TechNine user group meeting on March the 23rd. It is the world premiere of this presentation, so grab your seat and be able to tell your grandchildren you were there!

The Veeam hardened repository
The hardened repository with immutable backups

There is a tremendous amount of new and enhanced capabilities in v11, both small and big ones. But this one stands out because no matter who you are, a small business or a fortune 500 global enterprise, we are all at risk from ransomware and wipers. And guess what, the Veeam hardened repository gives you yet another option in your arsenal to make sure you can recover from such a disaster.

And end to end solution

In VBR v11 Veeam now offers complete end to end immutability for all your backup copies if you desire to do so. You can bring your own storage and so it leaves you the flexibility and freedom of choice you are used to with Veeam.

Like many of you, I like to stay up to date with new technology and implement it to solve real-world problems. Often coo tech is very forward-looking and foundational. This translates into “it is very early days yet”, “the code exists but you cannot use it yet”.

The Veeam hardened repository
You cannot delete immutable backups

Well, this is not something that shows you the promises of tomorrow and then leaves you to go back empty-handed. That always feels like a bit of a letdown when you are dealing with your real-world, real-time issues. The Veeam hardened repository is available now to help address the challenges you are facing today! It doesn’t get any better than that. I will show you the cookie, you can have the cookie and you can eat the cookie! Want to play with it yourself? You can, right now with the free community edition of Veeam Backup & Replication v11!

Register!

What do you need to do? Register of cause! Do it right here. Make sure you show up in time and stay around for an interactive discussion afterward. We start at 20:00 on March 23rd, 2021. And yes it is all virtual as we plan to get rid of the coronavirus first and then get together in real life again.

The Veeam hardened repository
register here

I will explain why you need it, how to set it up, and I will dive under the hood to show how it gets the job done. You can now protect your hypervisor and supported backups end to end with immutability in the hardened repository, giving you the extra security and protection you might need to recovery from a ransomware event. There will be demos! You can read up on it here but I will go behind the scenes with you!

Immutability of Linux files on the Veeam hardened Linux repository

Introduction

Without any doubt, I find the hardened Linux repository Veeam introduces in Veeam Backup & Replication v11 one of the most fascinating new features to get my hands on. In the ever-escalating battle with ransomware and wipers, this is a very valuable option to have in your defensive arsenal. So, I grabbed the Beta 2 and got to work in the lab over the holidays to investigate and find out some details about the Immutability of Linux files on the Veeam hardened Linux repository.

Immutability of Linux files on the Veeam hardened Linux repository

I’s quite easy to find the file attribute “i” that marks a file as immutable.

attr -a -l attr -a -l 
Immutability of Linux files on the Veeam hardened Linux repository
lsattr -a also shows the hidden files

or

attr -a -l
Immutability of Linux files on the Veeam hardened Linux repository
lsattr -a -l list out the full name of the attribute.

Where is the information about the immutability actually stored? I mean, that “i” attribute is one thing but how do the Linux host and Veeam know from what time period this immutability is valid. In the end, the service has to clear it and know when to do this. Or is this only stored in the Veeam database or both?

How does it now from when till when a file must be immutable?

Digging around in the files and folders of the Veeam repository, I soon found the lock file “.veeam.x.lock” (see the green arrow in the image above) that is created by the veeamimmureposvc service. The owner is root, hence it is not created by the Veeam transport service. The veeamimmureposvc service is a local account with root access for managing the immutability. It only works locally and does not listen on any network port, hence it cannot be accessed remotely.

Immutability of Linux files on the Veeam hardened Linux repository
The veeamimmureposvc service controls the .veeam.x.lock file. the x is a number has increments with every backup job you run.

Let’s look inside to see if we can read something there?

cat .veeam.9.lock
the lock file is an XML file containing all the date/time stamps for every file in that backup job.

When you open that file you will find it to be an XML file. Inside you’ll see the date and time stamp for every file in the backup chains for that job. That’s cool.

But there is more. When we run to look for extend file attributes we find that every Veeam created file has a one called user.immutable.until.

The backup files all have an extended file attribute called user.immutable.until.

With that name, it is clear it can be of interest to us. If you look at what is in there, you’ll see it contains the date and time stamp for that file’s immutability period.

getfattr * -n user.immutable.unil 
Immutability of Linux files on the Veeam hardened Linux repository
The extended file attribute contains the timestamp until when that backup file is immutable!

That I find interesting. Veeam saves the information twice. Is that for redundancy or as some sort of checksum? Maybe it also has to do with the fact Veeam backup files are transportable and self-contained so that information is stored as an extended file attribute.

Conclusion

So there you have it. A small piece of information on where the immutability information is stored. The most surprising thing to me was that it is actual stored twice.

I hope you fund this interesting. Poking around to figure out the how and what of things always helps me tremendously to learn and understand the technologies I want to work with. That leads to better decisions in design and implementation. It leads to both trust and confidence, which helps me decide where and when to leverage it. Finally it also, almost without, it is invaluable when supporting the technology.

Linux Integration Services Version 3.5 for Hyper-V Available For Download

Yesterday, December 19th 2013, Microsoft made the Linux Integration Services Version 3.5 for Hyper-V available for download.

The Linux Integration Services (LIS) package downloaded from Microsoft  is meant to deliver support older Linux distros. In the most recent Linux distros the KVP component is to be included, as are the other Hyper-V related drivers. In these distros these drivers and components are to be part of the upstream Linux kernel, and as such are included in Linux distros releases. So you should not need this download if you run these newer distros that has the LIS built-in. The list of supported distros is slowly growing.

image

If you are running (or need to run) older versions of Linux in your VMs and leverage the 100% fully featured Hyper-v Server 2012 R2 that is also 100% free of charge this is your way to leverage all those features. The aim is that you’re never a left behind when running Hyper-V (within the limits of supportability, DOS 6.0, NT 4.0 or Windows 2000 is not an acceptable OS today).

In Microsoft speak:

Hyper-V supports both emulated (“legacy”) and Hyper-V-specific (“synthetic”) devices for Linux virtual machines. When a Linux virtual machine is running with emulated devices, no additional software is required to be installed. However, emulated devices do not provide high performance and cannot leverage the rich virtual machine management infrastructure that the Hyper-V technology offers.

To make full use of all benefits that Hyper-V provides, it is best to use Hyper-V-
specific devices for Linux. The collection of drivers that are required to run Hyper-V-specific devices is known as Linux Integration Services (LIS).
 
For certain older Linux distributions, Microsoft provides an ISO file containing installable LIS drivers for Linux virtual machines. For newer Linux distributions, LIS is built into the Linux operating system, and no separate download or installation is required. This guide discusses the installation and functionality of LIS drivers on older Linux distributions.

For some extra info an tips see Enabling Linux Support on Windows Server 2012 R2 Hyper-V