mtellin.com

Removing VM snapshots left by CommVault or NetApp

Since I tend to focus on CommVault and NetApp this post is mentioning those two products in particular, but if you’ve used any VM level backup product that quiesces the VM prior to taking the backup – you have likely seen that these VMware snapshots aren’t always removed by the application. I’ve seen a few different ways to handle this, but since I am a beginner at PowerShell I thought I’d see how I could remove them without using the vSphere Client.

This all started due to a power outage over the weekend in the middle of a VMware backup so I was left with a number of VM’s with the all too familiar ___GX_BACKUP___ snapshot name which I determined by running the following command:

Get-VM | Get-Snapshot | Select Name,VM,SizeMB

Output from PowerCLI showing the existing VM snapshots

This part didn’t tell me anything that I didn’t already know, I have a number of VM snapshots that are left behind from CommVault all with the same snapshot name. In this particular case I had a backup job that failed, so it had already created the snapshots but hadn’t done the backup which also meant it hadn’t yet come through and removed the VM snapshots. Next up was running the following command to delete all of those snapshots:

Get-Vm | Get-Snapshot -Name __GX_BACKUP__ | Remove-Snapshot -Confirm:$false

Snapshots being committed

This should kick off a Remove Snapshot job if you have the vSphere Client open. Let it run and you should have all of your VM snapshots initiated by CommVault committed to disk.

If you are using NetApp SnapManager for VI to backup your virtual machines, your snapshot name will be different. Instead of ___GX_BACKUP___ it will be something along the lines of smvi_<string of numbers>. To remove those snapshots you can run the following:

Get-Vm | Get-Snapshot -Name smvi_* | Remove-Snapshot -Confirm:$false

Keep in mind that will delete ALL snapshots that start with smvi_, if you have valid snapshots you’d like to keep that were taken by SMVI, that command isn’t for you. That should be it, this is pretty basic but I seem to always have to deal with leftover snapshots so I wanted to document this for easy reference.

Popularity: 8% [?]

CommVault SnapProtect with NetApp NFS Volumes

Introduction

One of the newer features of CommVault Simpana is the ability to perform SnapProtect backups for various applications such as Exchange, SQL or VMware. This post will focus on VMware and specifically on virtual machines that are on a NFS datastore hosted on a NetApp storage array.

If you aren’t familiar with SnapProtect it allows you to take hardware based snapshots called from CommVault. Being a NetApp reseller, I would compare it to providing SnapManager-like functionality that you can control from within the CommCell console. If you are already a CommVault shop I think there are some pretty compelling reasons to take a look at it (and if you are currently licensed by capacity then you really should be looking at it as there is no extra charge). If you’ve seen CommVault’s demo video of taking a backup of 500 virtual machines in 17 minutes they are taking advantage of this functionality. The only thing I would point out is if SnapProtect is your only method for backing up your virtual machines, all you are getting is a hardware based snapshot for a backup. Ideally you should configure a backup copy to pull that data from the snapshot off the storage array and into your magnetic library within CommVault and ideally replicate it over to a disaster recovery location as well. All of that is possible within CommVault, it just requires a few extra things to configure in addition to basic SnapProtect functions.

Configuration

One of the things we ran into immediately when configuring this was an issue over whether NFS was even supported (yet) with SnapProtect. We were originally implementing this shortly after v9 was released and there wasn’t yet a lot of documentation on it. NFS is fully supported as a datastore type for SnapProtect backups and hopefully this post will help with a few of the stumbling blocks you may hit while configuring it.

Initial SnapProtect error

The first challenge I hit was network related. I installed the Virtual Server Agent on my Media Agent, which is on the server network VLAN (for the purposes of this post, I’ll use VLAN 10). We have another VLAN specifically for NFS/IP storage traffic used in our vSphere environment (again, for the purposes of this post I’ll use VLAN 20). Initially the media agent wasn’t configured on the storage VLAN 20 and the screenshot to the right shows the initial error I ran into. Simple enough solution, the network link from the media agent to the switch was configured as a trunk link with access to VLANs 10 and 20.

Array Management Control Panel

Now that I had the media agent configured with a storage VLAN my next issue was it was still trying to use the server VLAN when it connected to the NetApp array. This ended up being caused by how I setup the NetApp array within the Array Management Control Panel in the CommCell. After removing the entry that was configured with a hostname and instead using the IP address of the storage VLAN on the NetApp I was able to successfully connect. Lesson learned was this isn’t just a communication channel between the media agent and the storage device, it will match the IP address listed in the Array Management Control Panel to the datastore IP address that is used within vSphere. One thing we didn’t run into, but that you should keep in mind, is if you are using IP aliases you would then need to add in multiple entries for the same NetApp host, one entry per IP alias. It will continue on through the list trying to find a match so the order of the arrays within the Array Management Control Panel isn’t important.

Current Limitations

Having followed NetApp’s best practice guide the page file was on a separate VMDK file that was stored on a separate datastore in the vSphere environment. Unfortunately today SnapProtect doesn’t support this, the VMDK files must all reside within the same datastore. You also don’t have the ability to exclude a VMDK file or datastore, something that I hope will be added in a future release. This is one area where NetApp’s SnapManager tool has an advantage as you can choose to not snap the volume used for pagefiles and keep your space used for backups to a minimum. I like that approach so I can keep the snapshot sizes as lean as possible, I don’t need to backup transient data like the pagefile. If you followed the (optional) recommendation in NetApp’s best practices for vSphere (TR-3749) you would need to storage vMotion it to the same datastore as the rest of the virtual machine files before continuing with SnapProtect.

Another limitation I’ve run into with NFS volumes and NetApp is that if you follow best practices and disable the setting for automatically exporting a volume copying the snapshot into your magnetic library will not work. When you configure the backup copy to CommVault’s magnetic library it creates a FlexClone of the volume and that is what it will use to bring the data in. Without the options nfs.export.auto-update value set to on it won’t have the necessary NFS permissions and will fail to mount that FlexClone into vSphere. After installing CommVault SP2 recently I no longer have this issue, although I don’t see it mentioned in the release notes for the service pack.

The SnapProtect features are relatively new and hopefully these items are on the roadmap to change as it really is impressive seeing array level snapshots being orchestrated by CommVault.

Popularity: 16% [?]