VMware Building Block Guide - Virtual Server Agent for VMware

Table of Contents

The Virtual Server Agent (VSA) for VMware enables you to perform backup and restores and manage virtual machine data in complex virtualized environments. VSA uses vStorage APIs for Data Protection (VADP) or VMware Consolidated Backup (VCB) methods for data protection, and can operate in mixed environments with both current and older virtual machine platforms, with support for all guest operating systems supported by VADP and VCB.

Backup Agent

Overview

vStorage APIs for Data Protection (VADP) introduces several benefits over the VMware Consolidated Backup (VCB) method of protecting virtual machines.

There are several requirements that must be met for VADP to function correctly. This document covers those requirements for both the Virtual Server Agent and VMware infrastructure.

Backup Transport Methods

The three methods utilized by VADP to transport backup data are defined here.

  1. SAN – Data is read directly from the storage where virtual machines reside. This is also referred to as LAN-free since no data is transferred over the network. The LUN(s) containing the virtual machine disks must be visible to the Virtual Server Agent server to support SAN mode backups. SAN mode is supported when using storage connected over Fibre Channel or iSCSI.
  2. Hot-Add – The Virtual Server Agent is installed on a virtual machine residing on an ESX Server. Hot-Add mode can achieve close to SAN mode performance if SAN is not available. The term Hot-Add refers to the way the backups are completed. In Hot-Add the data volumes containing the virtual machines to be included in the backup are automatically mounted to the virtual machine.
  3. NBD & NBDSSL - NBD (network block device) & NBDSSL (encrypted NBD) transmit data over the ESX Servers' TCP/IP connection. This can be the production network or a dedicated backup network.

Refer to the Configuring Transport Modes topic for additional information and instructions on configuring transport modes.

Installation Requirements

System Requirements

For complete details on the system requirements for VADP environments, see System Requirements - Virtual Server Agent for VMware

Environmental Requirements

The vCenter server and all ESX Servers should be at vSphere 6.0, 5.x, 4.1 or 4.0 update 2 if possible. Should update 2 on 4.0 not be available, then a common user must be created on the vCenter server and all ESX Servers to avoid vmx file download issues.

  • VMware ESX Server Level - ESX(i) servers must be at version 4 or above.
  • Disk Types - Independent disks and Physical RDM’s are not supported. Virtual RDM’s are supported.
  • Stand Alone ESXi - Stand-alone ESXi is not supported without the standard license level. License levels are explained in more detail in the following article. http://blogs.vmware.com/esxi/2009/06/esxi-vs-esx-a-comparison-of-features.html
  • Fault Tolerant virtual machines - Fault Tolerant VMs are not supported for backup.

Port Requirements

In an environment with firewalls, the vCenter, ESX servers, Virtual Server agent, and MediaAgent must be able to communicate with each other. To ensure that all components can communicate through the firewall, ensure that the ports for web services (default: 443) and TCP/IP (default: 902) are opened for communication on each of these machines. Additional port requirements can apply for specific features such as Live Browse, Live File Recovery, and Live Mount. For additional information, see Entering Required Firewall Settings.

Prior to performing any backup or restore operations, ensure that the following port requirements are met. 

If a non-default port is used with VDDK 5.5, backup or restore operations can fail. See KB Article VMW0013.

Virtual Center

  • Port for web service (default: 443) must be opened. If vCenter is configured to use non-default ports, the non-default ports must also be opened.

ESX Server

  • Ports for web service (default: 443) and TCP/IP (default: 902) must be opened for the vStorage APIs for Data Protection.

vCloud Director

  • A port for the vCloud REST API (default: 443) must be opened.

Subclient with multiple proxies

  • If you are using multiple proxies in a single subclient, open the CVD port (by default 8400) for all proxies in the subclient, to enable the coordinator node and other proxies to communicate. The CVD port is used to establish communication between proxies using a randomly selected port; to enable communication between proxies you must configure the Simpana firewall.

Change Block Tracking Requirements

Change Block Tracking (CBT) must be enabled to support VADP backup. The Virtual Server Agent agent automatically checks and enables CBT at the time of backup. The Virtual Server Agent might not be able to enable CBT for cloned and migrated virtual machines. CBT can be checked by following the steps documented in the following VMware article: http://kb.vmware.com/kb/1020128.

VDDK Installation

The VDDK (Virtual Disk Development Kit) is now pushed with the installation of the Virtual Server Agent. No separate installation is required.

Hardware Requirements

The following hardware requirements are for Virtual Server Agent (VSA) installed on either physical or virtual machines. For more information on CommCell level hardware specifications, see CommCell Sizing.

Important:

  • If VSA is installed on the MediaAgent (physical or virtual machine), then the capacity of the MediaAgent that exceeds the VSA capacity can be used for other network based agents including remote VSAs up to the maximum capacity referenced in Hardware Specifications for MediaAgent.
  • For virtual machines, to achieve maximum performance and scalability, reserve the recommended CPU, memory, and other resources for Simpana software processes so that they operate within the set constraints.
Extra Large

Supports 40 to 50 TB of FET

  • 16 CPU cores
  • 32 GB RAM
Large

Supports 15 to 40 TB of FET

  • 8 CPU cores
  • 16 GB RAM
Medium

Supports 5 to 15 TB of FET

  • 8 CPU cores
  • 8 GB RAM
Small

Supports up to 5 TB of FET

  • 4 CPU cores
  • 4 GB RAM

I/O Paths:

  • Dedicated I/O to Datastores and backup disk.
  • Dedicated I/O should have one interface for read operations and another for writing to the backup disk.
  • The server should have recommended IOPS for deduplication operations as described in Hardware Specifications for MediaAgent.

Permissions for Custom User Accounts

You can create a separate user account in vSphere for backup and restore operations. When you create a user account, the following system permissions are automatically added to the account:

Category Permission
System Anonymous

Read

View

If you are creating a user account other than administrator, permissions can be assigned to the role associated with the user account. The following table shows which vCenter permissions are required (√) for each Simpana® role or component.

  • To enable restores, assign both backup and restore permissions for the type of restore (from streaming or IntelliSnap backups).
  • By default this list shows settings for vSphere 5.x; but differences for vSphere 4.1 are noted. Settings that are not available in vSphere 4.1 may be needed for features that require vSphere 5.0 or greater.
  • When using VM File Recovery Plug-In, VM Provisioning, or Live Mount, assign any required permissions for backups or restores as well as permissions for using that feature.
  • Live Recovery operations using a File Recovery Enabler for Linux require the same permissions as IntelliSnap operations.

Assign permissions for the following categories:

Disclaimer: The guidance here is derived from information published in vSphere Security: ESXi 6.0 and vCenter Server 6.0. For detailed and current information about vSphere privileges and permissions, refer to the appropriate VMware documentation. Commvault is not responsible for, and does not validate or confirm, the correctness or accuracy of any information provided here. All content in this section is provided "AS IS" and is not warranted by Commvault in any way.

Datastore Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Allocate space

Required to allocate space for a virtual machine, snapshot, clone, or virtual disk.

 
Browse datastore

Required to browse files on a datastore. Used to locate VM files on disk and verify that files exist.

 
Configure datastore

Required to configure a datastore.

 
Low level file operations

Required to perform read, write, delete, or rename operations for the datastore. Used to read virtual machine configuration files.

 
Remove datastore

(deprecated) Required to remove a datastore. The user or group privilege must be set for both the object and its parent object.

 
Rename datastore

Required to change the name of a datastore.

 
Remove file

(deprecated; use Low level file operations) Required to delete files in the datastore.

 
Update virtual machine files

Required to update virtual machine file paths on a datastore after a datastore resignature operation.

 

Extension Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Register extension

Required to register a plug-in.

 
Unregister extension

Required to unregister a plug-in.

 
Update extension

Required to update a plug-in.

 

Global Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Cancel task

Required to cancel a running or queued task; used to cancel a relocation task if a restore job is killed.

 
Diagnostics

Required to get lists of diagnostic files, log headers, binary files, or diagnostic bundles. For security, limit this privilege to the vCenter Server Administrator role.

 
Disable methods

Required to disable specific operations on vCenter entities.

 
Enable methods

Required to enable specific operations on vCenter entities.

 
Licenses

Required to view installed licenses and to add or remove licenses.

 
Log event

Required to enable logging of user-defined events against a managed entity.

 
Manage custom attributes

Required to add, remove, or rename custom field definitions. Used with the EnableUUID attribute to enable application-consistent quiescing.

 
Set custom attribute 

Required to view, create, or remove custom attributes for a managed entity. Used with the EnableUUID attribute to enable or disable application-consistent quiescing.

 

Host - Configuration Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Advanced settings

Required to set advanced options for host configurations. For live file recovery, Commvault software increases the NFS heartbeat timeout and the Max failures setting for additional resilience when mounting the datastore on the ESX server.

 
Connection

Required to change the connection status of a host (connected or disconnected). Used to confirm whether the ESX host is connected within the vCenter inventory.

 
Storage partition configuration 

Required for management of VMFS datastores and diagnostic partitions. This privilege enables users to scan for new storage devices and manage iSCSI. Used to rescan and check for new VMFS partitions and HBAs, and to refresh the datastore list when mounting a datastore to the ESX server during IntelliSnap operations.

 
System Management

Required to manipulate files on the host. Used to enable CBT in the VMX file and to make changes to the VMX file during restores.

 

Network Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Assign network

Required to assign a network to a virtual machine. Used to create a virtual machine on a network.

     

Resource Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Assign vApp to resource pool 

Required to assign a vApp to a resource pool during restores.

   
Assign virtual machine to resource pool

Required to assign a virtual machine to a resource pool. Required when registering a virtual machine to a resource pool during backups or when restoring to a resource pool.

√ (VM Archiving)  
Migrate powered on virtual machine

Required to use vMotion to migrate a powered on virtual machine to a different resource pool or host.

   
Migrate powered off virtual machine ("Migrate" in vSphere 4.1) 

Required to use vMotion to migrate a powered off virtual machine to a different resource pool or host.

 

vApp Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Import

Required to import a vApp into vSphere.

 
vApp application configuration

Required to modify vApp application properties; used when reconfiguring an existing File Recovery Enabler for Linux.

             
vApp instance configuration

Required to modify a vApp instance; used when reconfiguring an existing File Recovery Enabler for Linux.

             

Virtual machine - Configuration Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Add existing disk

Required to add an existing virtual disk to a virtual machine.

 
Add new disk

Required to create a new virtual disk to add to a virtual machine.

 
Add or remove device

Required to add or remove any non-disk device. Used to add a SCSI controller or to restore non-disk device configuration.

 
Advanced

Required to add or modify advanced parameters in a virtual machine configuration file.

 
Change CPU count

Required to change the number of virtual CPUs during restores.

 
Change resource

Required to change the resource configuration of a set of virtual machine nodes in a given resource pool.

 
Disk change tracking 

Required to enable or disable change tracking for virtual machine disks.

 
Disk lease

Required to perform disk lease operations for a virtual machine.

 
Display connection settings (not in vSphere 4.1)

Required to configure virtual machine remote console options.

 
Extend virtual disk

Required to expand the size of a virtual disk.

 
Host USB device

Required to attach a host-based USB device to a virtual machine.

 
Memory

Required to change the amount of memory allocated to a virtual machine.

 
Modify device settings

Required to change the properties of an existing device.

 
Raw device

Required to add or remove a raw disk mapping or SCSI pass through device (overrides other privileges for modifying raw devices, including connection states).

 
Reload from path

Required to change a virtual machine configuration path while preserving the identity of the virtual machine; used during failover and failback operations.

 
Remove disk

Required to remove a virtual disk.

 
Rename

Required to rename a virtual machine or modify notes for a virtual machine.

 
Reset guest information

Required to edit the guest operating system information for a virtual machine.

 
Set annotation (not in vSphere 4.1)

Required to add or edit a virtual machine annotation. Used to set up a backup server annotation that records last backup times for target VMs in vSphere.

 
Settings

Required to change general virtual machine settings.

 
Swapfile placement

Required to change the swapfile placement policy for a virtual machine.

 
Unlock virtual machine

Required to decrypt a virtual machine.

 
Upgrade virtual machine compatibility ("Upgrade virtual hardware" in vSphere 4.1)

Required to upgrade a virtual machine’s compatibility version (virtual hardware version).

 

Virtual Machine - Guest Operations Permissions (Not in vSphere 4.1)

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Guest Operation Modifications

Required to perform virtual machine guest operations that modify the guest operating system, such as transferring a file to the virtual machine or restoring files to a target VM that does not have a file system agent installed.

 
Guest Operation Program Execution

Required to perform virtual machine guest operations that execute a program in the virtual machine, such as a restore command.

 
Guest Operation Queries

Required to perform virtual machine guest operations that query the guest operating system, such as listing files in the guest operating system. Used when the target VM does not have a file system agent installed.

 

Virtual Machine - Interaction Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Device connection

Required to change the connected state of virtual machine devices that can be disconnected.

         
Power Off

Required to power off the guest operating system of a powered on virtual machine. Used when restoring data to VMDKs.

√ (VM Archiving)  
Power On

Required to power on a powered off virtual machine or resume a suspended virtual machine.

 
Reset

Required to reset a virtual machine and reboot the guest operating system.

 
Suspend

Required to suspend a powered on virtual machine and put the guest in standby mode.

 

Virtual Machine - Inventory Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Create new

Required to create and allocate resources for a virtual machine.

 
Create from existing

Required to create a virtual machine, by cloning an existing virtual machine or by deploying from a template.

 
Move

Required to relocate a virtual machine in the hierarchy. The privilege must be set for both the source and the destination.

√ (VM Archiving)  
Register

Required to add an existing virtual machine to a vCenter Server or host inventory. Required for IntelliSnap backups with metadata collection enabled, and to register a restored VM with the vCenter or host.

 
Remove

Required to delete a virtual machine and remove the underlying files from disk. The user or group privilege must be set for both the object and its parent object. Required for IntelliSnap backups with metadata collection enabled.

 
Unregister

Required to unregister a virtual machine from a vCenter Server or host inventory. The user or group privilege must be set for both the object and its parent object. Required for IntelliSnap backups with metadata collection enabled, and to unregister a VM so that it can be registered to a different location.

 

Virtual Machine - Provisioning Permissions

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Allow disk access

Required to open a disk on a virtual machine for random read and write access. Used for remote disk mounting and restoring data.

 
Allow read-only disk access

Required to open a disk on a virtual machine for random read access; used for remote disk mounting.

 
Allow virtual machine download

Required for read operations on files associated with a virtual machine, including vmx, disks, logs, and NVRAM.

 
Clone template

Required to clone a template.

 
Clone virtual machine

Required to clone an existing virtual machine and allocate resources. Used to create a linked clone from a source VM snapshot during backup.

 
Customize

Required to customize a virtual machine’s guest operating system without moving the virtual machine.

 
Deploy template

Required to deploy a virtual machine from a template.

 
Mark as template

Required to mark an existing powered off virtual machine as a template. Used to restore a virtual machine template.

 
Mark as virtual machine

Required to mark an existing template as a virtual machine.

 
Modify customization specification

Required to create, modify, or delete customization specifications.

 
Promote disks

Required to promote operations on virtual machine disks.

 
Read customization specifications

Required to read a customization specification.

 

Virtual machine - Snapshot management Permissions ("Virtual machine - State" in vSphere 4.1)

Permissions Streaming IntelliSnap and Live Recovery Streaming and IntelliSnap VM Lifecycle Management Provisioning Live Mount
Backups and VM Archiving Restores and Live Sync Backups Restores Deploy File Recovery Enabler for Linux VM File Recovery Plug-In
Create snapshot

Required to create a snapshot from a virtual machine’s current state.

 
Remove Snapshot

Required to remove a snapshot from the snapshot history.

 
Rename Snapshot

Required to change the name or description of a snapshot.

     
Revert to snapshot

Required to set a virtual machine to the state it was in for a specified snapshot.

 

Deployment Scenarios

The Virtual Server Agent can be installed in several ways to suit environmental needs and resources. Additionally, VADP installations now include VDDK (Virtual Disk Development Kit); no separate installation of this package is required.

The following sections describe frequently used deployment scenarios:

Physical Proxy Computer Installations

In this configuration, the Virtual Server Agent and MediaAgent are installed on a a physical server for SAN only backups with no data transferred over the LAN. A pure physical implementation will often provide the best performance and requires physical hardware with visibility into the storage network.

When to Use

  • Data Stores are configured on Fibre Channel or iSCSI SAN (No NFS).
  • Able to provide Physical Windows server with SAN access to Data Store LUNs
  • Direct tape copies are required or physical MAs required for secondary operations.

When NOT to Use

  • SAN access to Data Store LUNs from outside ESX Server environment is not possible.

Hot-Add Installations - Virtualized Agent and Virtualized MediaAgent

In this configuration, the Virtual Server Agent and MediaAgent are both installed in Hot-Add mode. The backup destination is typically network based (CIFS/NFS) or another vmfs datastore and tape out options are very limited. Shared storage is required for HotAdd mode backups of virtual machines living on other ESX Server hosts.

When to Use

  • All virtual environment where physical servers are not preferred.
  • Data Stores are configured on NFS.
  • Each MA can write directly to a mount point presented to the virtual machine
  • Able to segregate backup and production traffic on the network.

When NOT to Use

  • Direct backup to tape is required. Backup traffic cannot be segregated from production traffic.

Hot-Add Installations - Virtualized Agent and Physical MediaAgent

In this configuration, the Virtual Server Agent is installed in hot-add mode while the MediaAgent is installed on a physical computer. Data is transferred over the LAN to the physical MediaAgent. This model also allows for the use of a centralized Windows MediaAgent and Linux MediaAgents. Shared storage is required for Hot-Add mode backups of virtual machines living on other ESX Server hosts.

When to Use

  • Direct backup to tape or Silo to cloud is required.
  • Segregation of backup and production data is possible.
  • Physical MA is required for other protection tasks and secondary operations.

When NOT to Use

  • Segregation of backup and production data is not possible.
  • SAN based DataStores on Fibre Channel or iSCSI are in use. (use all physical configuration)

Installing the Agent

The Virtual Server Agent can be installed using the steps described in Getting Started - VMware Deployment.

Configuration

In most environments, the initial configuration of the Virtual Server Agent will include configuring backups of the vCenter. The following sections provide details specific to configurations in VADP environments.

VADP-Only Configurations

During backups, the Virtual Server Agent creates a snapshot of the virtual machine data directly from the datastore. The snapshot is then moved directly to the storage media without requiring any dedicated disk cache on the proxy server. In the case of  incremental backups, Change Block Tracking (CBT) helps quickly identify the data blocks on the virtual machine that have changed since the last backup.

Similarly, during restores, virtual machines are restored directly to the appropriate ESX Server and datastore without the need for staging on the proxy server with VMware Converter. This approach provides much faster restores.

Combined VADP and VCB Configurations

 To perform backup operations from an environment where both technologies exist, ensure the following components are installed on the proxy computer:

  • VMware Consolidated Backup (VCB)
  • vSphere VADP
  • Virtual Server Agent

Once these components are prepared, backups will leverage between VCB and vSphere as appropriate.

The vStorage API can be used to restore backups performed with VCB. The backup data will first be staged to the proxy and then vStorage will perform the restore.

Configure the Instance, Backup Set, and Subclient

Refer to Getting Started - VMware Configuration complete step-by-step instructions on configuring the Instance, Backup Set, and Subclient.

Configuration Considerations

  • Regular backups should be performed on the default subclient. This will ensure new virtual machines are discovered on a regular basis along with the backup operation.
  • Adding one virtual machine per subclient is not recommended. This creates issues around scheduling and becomes unmanageable in larger environments.
  • Typically only the default Subclient will be used in most deployments except in the following cases:
    • Multiple Storage Policies are required for different virtual machines.
    • Rule-based discovery is in use and different subclients will be used to target a particular affinity.

Backup

Once the Virtual Server Agent is configured, backup and restore operations are performed as described in Getting Started - VMware Backup.

Perform a Backup

After configuring the virtualization client and a subclient, you are ready to perform your first backup.

The following section provides step-by-step instructions for running your first full backup of a single virtual machine immediately.

  1. From the CommCell Console, navigate to Client Computers | <vCenter Client> | Virtual Server | VMware | <BackupSet>.

    Right-click the <Subclient> and click Backup.

  2. Select Full as backup type and Immediate to run the job immediately.

    Click OK.

  3. You can track the progress of the job from the Job Controller window of the CommCell console.

  4. Once job is complete, view the details of job from the Backup History.

    Right-click the Subclient and select Backup History.

  5. Click OK.

  6. You can view the following details about the job by right-clicking the job:
    • Items that failed during the job
    • Items that succeeded during the job
    • Details of the job
    • Events of the job
    • Log files of the job
    • Media associated with the job

Restore

Container Level Restore

A container level restore restores the entire virtual machine disk. Refer to Getting Started - VMware Restore for the steps required to perform this restore following your first full backup.

File and Folder Level Restore

Files and folders can also be restored from a virtual machine backup. The Advanced - VMware Restore topic includes information on the different restore types available.

Backup Agent Considerations

Using the Virtual Server Agent in HotAdd mode configurations

  • When deploying the Virtual Server Agent, install the software on a datastore with the largest VMFS block size. This is necessary to ensure that the Virtual Server Agent can mount and back up virtual machines residing on all datastores.
  • Helper virtual machines are not required for HotAdd Virtual Server Agent servers using VADP.

Changed Block Tracking

Changed Block Tracking (CBT) is a VMware feature that can be used to optimize backups of virtual machines by reading only the allocated and modified portions of a virtual disk. CBT is automatically enabled for virtual machines running on hardware version 7 or higher.

For more information on Changed Block Tracking, see Changed Block Tracking (CBT) on virtual machines (1020128).

IntelliSnap Backup

IntelliSnap Overview

The IntelliSnap backup enables you to create a point-in-time snapshot of the data to be used for various data protection operations.

IntelliSnap backup works in conjunction with software and hardware snapshot engines to provide snapshot functionality for data protection operations. A dedicated ESX Server can be used for selective copy to tier 2 storage. This completely removes any utilization on the ESX Server farm. Granular Recovery can be performed at this stage for individual file and folder recovery.

Advantages of IntelliSnap for Virtual Machine Protection

IntelliSnap offers significant advantages for protecting critical virtual machines (VMs):

  • Point-in-time snapshots are taken in a fraction of the time needed for traditional streaming backups, using VMware software snapshots to create hardware snapshots on storage arrays.
  • Because the time applications need to be quiesced for backups is minimized for snapshots, the time needed to reconcile the production VM with transactions that occur during the backup is also minimized.
  • Snapshots can be taken as often as necessary, providing multiple daily recovery points for high-transaction VMs.
  • Critical virtual machines can be recovered and restored to service quickly.
  • Full VMs or files and folders can be restored from snapshots or backup copies.
  • Snapshots can be backed up to disk by a proxy (backup copy), limiting the impact of backups on production VM resources.
  • Based on policies, snapshots are automatically catalogued and indexed for restores and Backup Copy.

For incremental backups, only changes are written to backup media.

Considerations

  • For less critical VMs, use traditional streaming backups with the VMware vStorage API for Data Protection (VADP), to provide daily backup and recovery with reduced infrastructure costs.
  • For special cases, use application or file system agents in guest VMs to manage backups. This approach is useful when storage is presented directly to virtual machines, such as raw device mappings (RDMs), direct iSCSI, or network file system (NFS); large databases or large numbers of files; and applications that are not snapshot compatible.

Assign Virtual Machines to Tiers Based on Service Level Agreements (SLAs)

To design your protection deployment for virtual machines, classify VMs according to the recovery point objectives (RPOs) for different types of VMs.

Tier 1

Tier 1 VMs are the most critical VMs:

  • First priority for recovery
  • Multiple recovery points per day
  • 1 TB or more in size with multiple disks
  • High transaction and data change rates
  • Dedicated resources

Recommended Protection Method: Use IntelliSnap backups for Tier 1 VMs.

Tier 2

  • Second priority for recovery
  • One or two recovery points per day
  • Moderate to low transaction rates
  • 200-500 GB in size with multiple disks

Recommended Protection Method: Whenever possible, use traditional streaming VADP backups. If necessary, schedule daily or twice daily IntelliSnap snapshots to support fast recovery. For special cases use in guest agents.

Tier 3

  • Lowest priority for recovery
  • One recovery point per day
  • Low transaction rates
  • Fairly static data

Recommended Protection Method: Use traditional streaming VADP backups with no snapshots.

Protection Methods

IntelliSnap Plus Backup Copy

When IntelliSnap is used, a hardware snapshot is created on the storage array as soon as the VMware snapshot is completed, then the VMware snapshot is removed immediately. This approach minimizes the size of the redo log and shortens the reconciliation process, to reduce the impact on the virtual machine being backed up and lessen the storage requirement for the temporary file. The secondary backup copy is then performed from the hardware snapshot, outside the production environment.

Large, critical, and high transaction virtual machines can be excluded from streaming backups and protected using IntelliSnap snapshots. Snapshots support application consistent backups for applications such as Oracle, SQL Server, SharePoint, and Exchange. Hardware snapshots provide multiple persistent recovery points a day for critical VMs, including full VM recovery as well as granular file and folder level recovery, while minimizing the load on production VMs and infrastructure.

In the event of corruption of a full datastore where the underlying storage volume is intact, IntelliSnap enables a full volume revert at the hardware level, enabling all the VMs in the datastore to be recovered at the same time, leading to lower down time.

Multiple readers can be configured to perform simultaneous quiescing of multiple VMs for faster creation or deletion of software snapshots, reducing the overall IntelliSnap job time.

The redo log time for large VMs is minimized, to enable larger datastores and VMs to be protected.

A snapshot (Snap Copy) can be used by an ESX proxy as the source for secondary VADP backups, offloading the backup load for the largest VMs to minimize the impact on production systems.

Benefits

  • Low impact on productions systems
  • Multiple recovery points per day
  • Fast recovery copy
  • Reduces the backup workload by using a proxy server to create the daily backup copy.

Considerations

  • An ESX proxy is needed to mount a hardware snapshot for Backup Copy operations.
  • Additional storage required for snapshot reserve.
  • Possible impact on other production systems using same storage.
  • Array overhead during snapshot mount operations.

Traditional VADP Protection

Backups of virtual machines begin by quiescing the virtual machine and taking a VMware software snapshot. While the backup is in progress the virtual machine disk (VMDK) is frozen and changes are written to a temporary file (redo log). Once the backup completes, the VMware snapshot is removed and the virtual machine disk is updated with changes from the redo log (reconciled). For high transaction VMs the redo log can grow significantly if the backup takes a long time to complete, approaching the size of the entire VMDK for the source virtual machine. For other VMs, the changes recorded in the redo log are minimal.

VADP provides protection through an initial full backup supplemented by daily incremental backups and a periodic synthetic full backup to provide a single daily recovery point. This approach can include the following elements:

  • VMware‘s Changed Block Tracking (CBT) feature identifies changed blocks for incremental backups.
  • A synthetic full backup merges incremental changes from backup media to create a complete point-in-time VM backup without accessing the production VM.
  • A secondary copy of a virtual machine can be updated regularly using the DASH Copy feature with deduplication. Only changed data is written to the secondary copy. DASH Copy creates a new full backup by updating reference counters on deduplicated blocks that exist on disk. DASH Copy can be run on a frequency dictated by your backup retention policy, ensuring that old data blocks that are no longer needed are deleted from the system (known as data aging).
  • Advanced transport modes for VMs (SAN or HotAdd) minimize the impact of backup operations on local area networks (LANs).

This approach meets short term and long term retention needs with minimal impact on VMs, and can be used when a limited number of ESX proxies (required for IntelliSnap Backups) are available.

Benefits

  • It is possible to recover a full VM or any individual file from an incremental point in time backup (primary copy or any secondary copies). There is no need to consolidate daily backups into a synthesized full backup.

Considerations

  • For high transaction VMs, this approach may not be optimal, because high change rates in VMs require a long reconciliation process.
  • For high transaction VMs, incremental changes using CBT can take almost as long as a full backup because changes are written across the disk.
  • If hardware snapshots are required for some virtual machines, those can be configured in a separate backup set.

In-Guest Agents

In some cases, high transaction virtual machines can be protected by installing an application or file system agent in the guest (source VM). This approach is useful when storage is presented directly to a virtual machine including RDMs, direct iSCSI, or NFS, or when the VM has a large database or large number of files. This can also be used to address VMs that do not tolerate a VMware snapshot no matter how brief the duration.

Best Practices for Implementing Virtual Machine Protection

Backup sets provide an organizational and management point for each class of VM. Use a separate backup set for each protection method. For example, use one backup set for Tier 1 VMs with IntelliSnap plus Backup Copy, and another for Tier 3 VMs with traditional VADP protection.

Use a small number of subclients per backup set, each including similar classes of virtual machines across datastores, with separate subclients for large, critical, and high transaction VMs. This approach provides greater scalability and simplifies management; it also requires less intervention as the environment grows and changes (for example, removing or adding datastores or moving a VM to a different datastore).

Organize high transaction VMs in separate datastores. This approach is recommended for application performance; it also enables the use of hardware snapshots for data protection without processing too many snapshots for each datastore. Do not include VMs that are backed up using traditional VADP on the same datastore as VMs that are backed up using IntelliSnap plus Backup Copy.

Use naming patterns and filters to automatically include or exclude datastores, hosts, and virtual machines in a subclient.

All virtual machines in a single datastore should be protected as part of the same storage policy to minimize the number of snapshots.

Scheduling Considerations and Examples

Use schedule policies to manage the timing of each type of protection operation.

Schedule IntelliSnap and streaming VADP protection operations at different times, to avoid contention issues at the hypervisor or storage level.

Schedule virtual server agent (VSA) and guest-based application agent protection operations at different times. Both protection methods create application consistent backups; they should run at different times to avoid any potential conflicts.

Examples

The following tables provide examples of how different classes of VMs and protection methods can be scheduled.

VM Class Tier 1 Tier 2 Tier 3
Protection Method IntelliSnap plus Backup Copy IntelliSnap plus Backup Copy Traditional Streaming VADP
Backup Frequency Every 6 hours Daily Daily
Copies 3 2 2
Copy Frequency Every 6 hours Daily Daily
Schedule Daily at 6 am, noon, 6 pm, midnight Daily at 3 am, 3 pm Daily at 8 pm
Backup Copy daily at 1 am Backup Copy daily at 4 am Synthetic Full daily at 10 pm
Retention Copy 1 – 7 days
Copy 2 – 30 days
Copy 3 – 60 days
Copy 1 – 30 days
Copy 2 – 60 days
Copy 1 – 7 days
Copy 2 – 30 days

ESX Proxy Sizing (SAN Only)

When deploying an ESX proxy in a setup where SAN transport will be used, the ESX proxy is used only to mount the datastore and register the virtual machine. Data moves over SAN transport to a physical Virtual Server Agent or MediaAgent. Ensure that the following requirements are met:

  • Dual Quad-Core processors
  • 16-32 GB RAM
  • 2 HBA ports dedicated to SAN connectivity
  • Dedicated Network Adapter providing connectivity to the management network

IntelliSnap User Permission Requirements

See Permissions for Custom User Accounts.

Snapshot Engine Requirements

Refer to Storage Arrays Configuration - VMware for information on the requirements and configurations for snap engines.

IntelliSnap Configuration

Enter the Array Information

Refer to Storage Arrays Configuration - VMware for information on the requirements and configurations for snap engines.

Ensure that the array information is entered in the same format that is presented to the ESX server. (for example – should storage be presented by IP to the ESX servers it must also be entered by IP in Array Management)

Configure IntelliSnap

Refer to Getting Started - VMware IntelliSnap Configuration for step-by-step instructions on performing the following configuration tasks:

  • Configuring the Instance, Backup Set, and Subclient
  • Creating a Snapshot Copy
  • Configuring the Backup Copy

Perform an IntelliSnap Backup

Refer to Backup - Virtual Server Agent for VMware for step-by-step instructions on performing a backup using IntelliSnap.

Perform an IntelliSnap Backup Copy

Refer to Getting Started - Snap Movement to Media for step-by-step instructions on performing a backup copy.

Restore

Container Level Restore

A container level restore restores the entire virtual machine disk. Refer to Getting Started - VMware Restore for the steps required to perform this restore following your first full backup.

File and Folder Level Restore

Files and folders can also be restored from a virtual machine backup. The Advanced - VMware Restore topic includes information on the different restore types available.

IntelliSnap Considerations

  • In environments leveraging Fibre Channel storage (required for HDS), install the Virtual Server Agent and MediaAgent on a physical computer.
  • In array management. enter storage addresses for iSCSI and NFS in the same format used for ESX servers. For example, use an IP Address for both entries.
  • When using NFS storage, enter each IP address that is used into array management. Entering only a single IP for a management interface is not sufficient.
  • The Virtual Server Agent proxy must have access to the storage network. If you have an isolated network, an additional network connection must be added to the proxy.
  • IntelliSnap performs a full backup. Switching to or from a IntelliSnap backup causes the next backup to be a full backup.