Optimize the Veeam preferred networks backup initialization speed

When Veeam preferred networks cause slow backup initialization speeds

When using preferred networks in Veeam you choose to use another than the default host network for backups and restores. In this post, we’ll discuss how to optimize the Veeam preferred networks backup initialization speed because we aim for optimal performance. TL-DR: You need to provide connectivity to the preferred networks for the Veeam Backup & Replication server. It seems a common mistake I run into every now and then. Ultimately it makes people think Veeam is slow. No, it is just a configuration mistake.

Why use a preferred network?

Backups can fill up a 1Gbps pipe very fast. Many people still use 1Gbps networking as default connectivity to the hosts. Even when they leverage 10Gbps or better it is often in a converged network setup. This means that only part of the bandwidth goes to host connectivity. Few have 10Gbps for “just” host connectivity. This means it makes sense to select a different higher bandwidth network for backup and restore traffic.

Hence for high volume, high-performance backup and restores it is smart to look for a bigger pipe to leverage. Some environments have dedicated backup networks at 10Gbps or better. But we find way more high bandwidth networks for other purposes. In Hyper-V environments, you’ll have those for SMB networking like CSV, Live Migration variants and storage replication. Hyper-Converged Infrastructure deployments use these networks for storage as well. With S2D you’ll find more and more 25/50/100Gbps. All these can be leveraged as a preferred backup network in Veeam

Setting up a preferred network

Setting up a preferred network is easy. First of all, you figure out which network to use. You then add those to the preferred networks as follows:

In file menu select “Network Traffic Rules”

Optimize the Veeam preferred network backup initialization speed

Click “Add” and specify the source IP as well as the target IP range. You can op to encrypt the traffic and /or set a bandwidth limit.

We have two SMB storage networks available, we enter both.

There is no need to have the preferred network registered in DNS. It will work fine without.

I hope it is clear that the source (Hyper-V Hosts), the target (backup repository or the extends in a Scale-Out Backup Repository) and any Off Host Proxies need connectivity to the preferred network(s). If you leverage WAN accelerators, Gateways Servers, log shipping servers than these also need access. Last but not least you should also make sure that the Veeam Backup Server (VBR) has access to the preferred networks. This is one that a lot of people seem to forget. May because it is most often a VM if it is not a shared role on the repository server or such and things do work without it.

When the VBR server has no access to the preferred networks things still work but initialization of the backup and restore jobs is a lot slower. Let’s test this.

Slow Initialization of backup and restore jobs

As a result of using preferred networks you might probably notice the following:

  • First of all, we notice a slow down in the overall initialization of the backup and restore job.
  • This manifests itself in a slow start of the actual VM backup/restore and reducing the number of simultaneous backups/restores of VMs within a job.

Without the VBR server having connectivity to the preferred networks

23:54 to complete the backup job (no connectivity to the preferred network)

Optimize the Veeam preferred networks backup initialization speed

With the VBR server having connectivity to the preferred networks. Notice how smooth and continuous the throughput is.

07:55 to complete the backup job (with connectivity to the preferred network) => 3 times as fast.

When you look into the Veeam backup logs for this job you will find at various stages attempts by the VBR server to connect to the preferred networks. If it can’t it has to wait until it times out. You see entries like:

A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.10.110.2:2509 (System.Net.Sockets.SocketException)

Optimize the Veeam preferred network backup initialization speed
Just a small part of all the NetSocket time out you will find for every single VM in the job. Here VBR is trying to connect to one of the extends in the SOBR.

This happens for every file in the backups (config files and disks) for every extend in the Scale-Out Backup Repository (per VM backup chain). This slows down the entire backup job tremendously.

Conclusion

I always make sure that the VBR servers in my environments have preferred network connectivity. Consequently, initialization is faster for both backups and restores. Test it out for yourself! It is the first thing I check when people complain of really slow backup. Do they have preferred networks set up? Check if the VBR server has connectivity to them!

10 thoughts on “Optimize the Veeam preferred networks backup initialization speed

  1. In a SOFS/Hyper-Converged S2D Setup, you can skip all this if you have initially added the proxy as SMB-Target, like \\Proxy\Repository. Then SMB Multichannel would do this automatically for you and go over the “big” SMB-pipes instead of the default networking… (A trick Veeam support staff told me :-)). But yeah, for that, as you explain, the Proxy should have that SMB connectivity also (where usually it’s a VM which would make it a bit more tricky to setup: You’d have to use vRDMA in the VM for that, connected to the host SMB/RDMA-NICs, but hey, works like a charm)

    • If it’s not using SMB3 protocol there is no need for RDMA on the VM, just the connectivity to the VBR VM will do to get the faster initialization with preferred networks. We actually leverage the available SMB networks in standard shared storage cluster for backups as well with or without off-host proxies. While it is most often the SMB network we use, it is leveraged for other traffic (TCP/IP) as well. There are other tricks to leverage a separate network for those as well without using preferred networks. Options, options.

  2. What if the repo and VBR are on the same network as the hyperv hosts? The repo and vbr have multiple NICs and but default is the same vlan. The Backup copy jobs are slow (atleast slow per file.. ~40MB/s on a 10GE line).
    Copyjob repo is on another subnet.
    No prefered networks defined..

    • This is a specific initialization issue. Your question is hard to answer without looking at the complete environment, it depends. Open a support call and let them take a look.

  3. Pingback: Make Veeam Instant Recovery use a preferred network - Working Hard In IT

  4. Pingback: Troubleshooting 100% stalled Veeam backup jobs - Working Hard In ITWorking Hard In IT

  5. Pingback: Backup Features of Veeam Backup and Replication v10 | StarWind Blog

  6. Pingback: Veeam Hardened Repositories on DELL R740XD2 | StarWind Blog

  7. Many thanks for sharing this. It helped a lot. I noticed the backup running smoothly after the VBR server connectivity to the preferred networks.

Leave a Reply, get the discussion going, share and learn with your peers.

This site uses Akismet to reduce spam. Learn how your comment data is processed.