Backup Workload Distribution

The Virtual Server Agent (VSA) dynamically distributes virtual machines for backup across available VSA proxies, both at the beginning of the backup job and while the backup is running. When a backup operation starts, the first VSA proxy listed for the Virtual Server instance acts as a coordinator, controlling traffic for backups across all proxies.

Dynamic distribution improves performance, enhances scalability, and ensures fault tolerance for backup operations by providing the following functions:

• Better coordination across multiple jobs or large numbers of subclients

• Adaptive dispatch logic

• Load balancing across the backup infrastructure

• Throttling for proxies and datastores

• Improved failover in the event that a proxy or coordinator becomes unavailable

You can assign proxies for the instance and subclient. The list of proxies for the subclient overrides the list for the instance.

To adjust the default behavior, see Performance Tuning for Backups.

Virtual Machine Backup Priority

Virtual machines are prioritized for backups based on the following criteria:

1. Newly added VMs are prioritized over existing VMs.

2. VMs that failed in a previous backup attempt are prioritized over VMs that were successful in the last attempt.

3. If fewer proxies are available to back up a VM, then the VM has higher priority than VMs where a larger number of different proxies could be used to perform the backup.

4. For VMs that have the same time for the last backup, VMs are prioritized based on the number of disks and the size of the disks:

   1. VMs that have more disks have a higher priority.

   2. VMs that have more used space have a higher priority.

   The VM size is calculated as the sum of the sizes for all VM disks.

If a proxy stops processing a VM, the VM status is reset to Not Started and the VM returns to the queue of VMs waiting to be backed up.

Stream Allocation

The Virtual Server Agent can use multiple streams to back up VMs or to back up multiple disks for a single VM, with VM backups taking precedence. Proxy stream throttling limits the maximum number of streams on each proxy. When assigning a stream to a proxy, the coordinator uses the following priorities:

1. Assign streams to unused proxies first. Give a stream to each available proxy before assigning multiple streams to any proxy.

2. When assigning multiple streams, assign streams to proxies with the lowest percentage of streams in use.

The coordinator also uses information about the current workload and the resources for each proxy to determine how many streams can be assigned to each proxy:

• At the beginning of a backup job, the coordinator checks for other backup jobs that are in progress.

• Each proxy reports the number of jobs, the number of streams in use, and the number of CPU cores and total random access memory (RAM) in MBs (or the stream limit if one is configured using the nStreamsPerCPU additional setting).

• The coordinator uses the memory and CPU information provided by proxies to calculate the number of streams each proxy can support.

  By default, each CPU in a proxy can support 10 streams, and each stream requires 100 MB of memory.

• Based on these proxy resources, the coordinator sets the stream limit for each proxy and allocates streams.

  When distributing streams, the coordinator distributes streams to a proxy until the proxy reaches its limit (considering all processes and not just the current job). When a proxy reaches its limit, the coordinator does not assign any more streams to the proxy until all proxies have reached their resource limit.

  When all proxies are at their resource limits, the coordinator assigns the remaining streams using round-robin distribution.

• Whenever the number of streams changes, the proxy notifies the coordinator.

You can control the number of simultaneous backup streams used by a subclient by selecting a value for the Number of Data Readers box (Advanced Options tab on the Subclient Properties dialog box).

You can limit the number of streams that all subclients under a virtualization client can use by selecting a value for the Maximum number of parallel data transfer operations option (Job Configuration tab on the Advanced Client Properties dialog box).

Failover

The Virtual Server Agent ensures that backups continue, even when a proxy or the coordinator fails:

• If a proxy fails, the backup job continues, and a VM that the proxy was processing returns to the list of unprocessed VMs for the backup job. The next active proxy that becomes available backs up that VM.

• If the coordinator fails, the backup job goes to pending state. The next active proxy on the proxy list becomes the coordinator and automatically restarts the backup job for any virtual machines remaining in the job. Any VMs that were already backed up as part of the interrupted job are not backed up again.