Automatic Failover ensures immediate access to critical virtual machines (VMs) in the event of an outage:
- Continuously monitors VMs that have been configured for Live Sync replication to a secondary site.
- Detects problems with VMs or network access for the primary site.
- Automatically fails over to the secondary site if the VMs in the primary site are disabled or unreachable.
- Brings destination VMs online immediately or after a specified delay.
This feature is available for the Virtual Server Agent with VMware.
Monitoring and Failover
To support automatic failover, you must identify three Virtual Server Agent (VSA) proxies that reside in the primary site, to monitor VMs in the source datacenter.
- The first proxy selected during configuration of automatic failover acts as a coordinator to monitor the source VMs.
- Other proxies in the source datacenter act as workers.
Each worker monitors a list of VMs, with the coordinator making assignments as well as monitoring its own list of VMs.
- Each worker sends heartbeat packets continuously to the VMs on the list that the worker monitors, and checks responses to determine whether each VM is available or down.
- Workers communicate with the coordinator to report the states of the VMs they monitor.
- When necessary, the coordinator initiates an unplanned failover to bring destination VMs online.
Before You Begin
- Configure Live Sync for virtual machines that need to be monitored.
- To enable automatic failover, create the bEnableVSAutofailover additional setting and set the value to true on all proxies.
For instructions on adding additional settings from the CommCell Console, see Add or Modify an Additional Setting.
Property Value Name bEnableVSAutofailover Category VirtualServer Type Boolean Value true
You can disable monitoring by setting the value for the bEnableVSAutofailover additional setting to false on all proxies, and re-enable monitoring by setting the value to true.
- Identify three machines in the source datacenter to be used for monitoring.
Note: If you define multiple failover groups to support automatic failover, assign a different set of VSA proxies for each failover group.
- Verify that all monitoring machines are DNS resolvable and can communicate with the CommServe system, with the virtual machines they are monitoring, and with each other.
- Install the following software on monitoring proxies:
- Windows Server 2008 R2 or later
- Virtual Server Agent
- Java Development Kit 1.8 or later, with the JAVA_HOME environment variable set
- Virtual machines being monitored must meet the following requirements:
- Each virtual machine must have a unique DNS name.
- VMware Tools must be installed on each VM.
You can configure automatic failover as part of creating a failover group.
- In the CommCell Browser, under Dev-Test & DR, right-click Failover Groups, and then click New Group.
The Failover Group dialog box appears.
- On the Enter the Group Name and Description page, name the policy, type a description, and then click Next.
- On the Source Virtualization Client page, from the Virtualization Client list, select a client, and then click Next.
- On the Select Machines page, identify the VMs to assign to the failover group:
- To select some of the VMs from the list, click the box next to the VM name in the Client Machine column.
- To select all of the VMs in the list, click Select All.
- Click Next.
- On the Failover Configuration page, set the priority, failure, and interval options:
- In the Virtual Machine list under Priority, for each VM, select the order in which you want the destination VMs to power on.
- To continue powering on VMs according to priority settings after a VM fails to power on, select Continue on Failure.
- To configure the time period to wait before powering on the next VM, next to Priority Interval, type or select the number of minutes.
- Click Next.
- On the AutoFailover page, enable automatic failover and identify monitoring nodes:
- Select Enable AutoFailover Capability.
- To specify a delay between detecting an outage and failing over, select the delay in Delay time before declaring outage.
- Click Add.
The Add monitoring machine dialog box appears.
The first proxy you select acts as a coordinator to monitor the source VMs.
- For locationType, select the location of the monitoring node: Source is the only supported value.
- From the client list, select a VSA proxy.
- Repeat these steps to identify additional monitoring nodes.
- Click Next.
- On the Summary page, click Finish.
Viewing Failover and Live Sync Status
- See Viewing Failover Group Information to view failover group information or view a report.
- See Monitoring Live Sync to view failover status for virtual machines.
If a monitoring proxy is unavailable or if the heartbeat monitoring service stops running on a proxy, you can restart the monitoring service by deleting the failover group and recreating it using the procedure in this page. To re-enable monitoring with the original failover group, perform the following steps:
- Download the Auto Failover Troubleshooting QScript and place it in the Commvault Base folder on the CommServe system.
- On all monitoring proxies, disable monitoring by setting the bEnableVSAutofailover additional setting to false.
- Log in to the CommServe system using qlogin.
- From the Base folder on the CommServe system, to get the virtual application ID (vappId) for the monitoring service, enter the following command with the name of the failover group as input:
qscript -f GetVirtualAppId.sqle -i "failover_group_name"
- On each proxy, go to the Commvault \Base\vmhertbeatmon\ folder and execute the following command, where vappid_value is the vappID returned by the preceding step:
vmheartbeatmon.exe -vappId vappid_value