Remove Lingering Backup Checkpoints from a Hyper-V Virtual Machine

Sometimes things go wrong with a backup job. There are many reasons for this, but that’s not the focus of this blog post. We’re going to show you how to remove lingering Backup checkpoints from a Hyper-V Virtual Machine that where not properly removed after a backup on Windows Server 2012 R2 Hyper-V.

You see checkpoint with a “odd” icon that looks like this

image

You can right click on the checkpoint but you will not find an option to apply or delete it.This is a recovery snapshot and you’re not supposed to manipulate this manually. It’s there as part of the backup process. You can read more about that in my blog posts Some Insights Into How Windows 2012 R2 Hyper-V Backups Work and What Is AutoRecovery.avhdx all about?.

When we look ad the files we see the traces of a Windows Server 2012R2 Hyper-V backup

image

Some people turn to manually merging these checkpoints. I have discussed this process in blog posts 3 Ways To Deal With Lingering Hyper-V Checkpoints Formerly Known as Snapshots and Manually Merging Hyper-V Checkpoints. But you don’t have to do this! As a matter of fact you should avoid this if possible at all. The good news is that this can be done via PowerShell.

I’m not convinced the fact that you can’t do it in the GUI is to be considered a bug as some would suggest. The fact is you’re not supposed to have to do this, so the functionality is not there. I guess this is to protect people from trying to delete one by mistake when they see one during backups.

Anyway, PowerShell to the rescue!

So we run:

Get-VMSnapshot -ComputerName "MyHyperVHost" -VMName "VMWithLingeringBackupCheckpoint"

#

This show us the checkpoint information (note it does not show you the XXX-AutoRecovery.avhdx” as a checkpoint.

image

And then we simply run Remove-VMSnapshot

#Just remove the lingerging backup checkpoint like you would a normal one via PoSh 
Get-VMSnapshot -ComputerName "MyHyperVHost" -VMName "VMWithLingeringBackupCheckpoint" | Remove-VMSnapshot

You can see the merging process in the GUI:

image

If you inspect the remaining MyVM-AutoRecovery.avhdx file you’ll see that it points to the parent vhdx.Normally your VM is now already running from the vhdx anyway. If not you’ll need to deal with it.

image

During a normal backup that file is deleted when the merge of the redirect AVHDX is done. You’ll need to do that manually here and be done with it.

Conclusion: there is no need to dive in and start merging the lingering backup checkpoints manually. Leave that for those scenarios where that’s the only option you have left.

15 thoughts on “Remove Lingering Backup Checkpoints from a Hyper-V Virtual Machine

  1. I would love this to work, but for me it doesn’t.
    I have some of this checkpoints left over from Acronis Backup jobs. I only get this error when trying to remove the checkpoints

    Remove-VMSnapshot : ‘NAME’ failed to remove checkpoint. (Virtual machine ID 4EDF5034-3260-46C7-B44F-885A7AAEF4DB)
    Cannot delete checkpoint: Catastrophic failure (0x8000FFFF). Checkpoint ID 1964A143-6EB9-40F6-B649-5C9ACD2C27FF.
    Virtual machine failed to generate VHD tree: ‘Catastrophic failure'(‘0x8000FFFF’).

  2. Flawless.
    My situation came from pulling a backup off of system center DPM into filesystem after a failed recovery over VM.

    Ended up deleteing corrupted vhds and copying over the recovered vhds through filesystem. In failover cluster, adjusted the vm config so disks point to the -AutoRecovery.avhds, OKed through a bunch of intimidating “you can corrupt the snapshot chains” messages, and then started the VM. Ran the powershell command to remove the snapshot and the merge happened so fast I had to double check the files to make sure they were gone, but it worked!

    Cheers m8

  3. I’ve got a very similar situation, which has been going on for around a week. I actually have a chain of recovery checkpoints, four of them!

    So is it still safe to run – Get-VMSnapshot -ComputerName “MyHyperVHost” -VMName “VMWithLingeringBackupCheckpoint” | Remove-VMSnapshot

    As I think it may attempt to remove the checkpoints in the wrong order, basically the order that powershell reports them with the “Get-VMSnapshot” command, which is earliest first.

    Any ideas? This is a production web server so I’m not happy about diving in and giving it a go. I have a backup, but I’ve no idea they’re OK given these recovery points – the backups could have been disrupted totally by the server reboot. 🙁

    Just for info, the reason I have four chained recovery checkpoints is because I accidentally set up a server reboot scheduled task (twice a week) at the exact same time in the evening as the Windows Server Backup (ran every evening). Every day the server reboots, it throws up another recovery checkpoint!

    Thanks!

    • If they are truely lingering checkpoints it should not matter. But I cannot determine this from here. What you can always do is create yet another checkpoint manually of the VM when it’s shut down (that’s “app’ consistent). You can export that checkpoint (even while the VM is running) your clean up action goes south all you need to do is import the VM again which is free of checkpoints. Good luck.

      • Many thanks. Good idea about taking another snap…

        I have tried the command out on another VM which was doing the same thing – this one a lot less critical. It had two such checkpoints.

        Seemed to work okay – I suspect S2012R2 Hyper-V is sufficiently developed enough for it not to be a problem.

        I’ve had other things on the go so haven’t yet done it on the production VM.

        Thanks!

      • I thought I’d do it.
        Confirmed working. You CAN remove a chain of recovery checkpoints using the powershell method above – at least using a bang up to date S2012 R2 server.

        It has been a couple of weeks since the checkpoints were made, so it took quite a time to merge them back in. The machine went a bit unresponsive in patches, but came back responsive before the merge finished up.

        Thanks!

  4. Any idea what causing this? Doesn’t matter if the server is online or shut down.
    “Remove-VMSnapshot : The operation cannot be performed while the virtual machine is in its current state. The name of the snapshot is xxxxx – (31/01/2017 – 17:59:33) and its ID is xxxxx. At line:1 char:66.”

    Also, how much free space does the merging require? The same as the VM’s actual disks or more? I have a VM with a chain of almost 30 checkpoints that’s taking almost all available space.

Leave a Reply