Unplanned Failovers in a Linux Environment

An unplanned failover can be performed in the event of a disaster, such as the following:

• When the production CommServe host is inaccessible due to network or hardware failure

• When the production CommServe host is partially available

• When there is a failure during a planned failover

Procedure

1. Perform the following steps, if the production CommServe host is available:

   1. Login to the production CommServe host.

   2. If the CommServe is currently in a "lockdown" mode where client certificate authentication is forced during installation, reset the CommServe lockdown by setting the Force per-client certificate authentication option to No from the CommCell Console > Control Panel > Certificate Administration dialog box. (This setting can be re-enabled once the failover is completed.) For more information on client certificates, see Client Certificates - Enforcing Authentication of Client Certificates during Installations.

   3. Shut down the services associated with both the CommServe and the SQL client in the production CommServe host.

2. Open command prompt and navigate to the folder associated with the SQL client:

   For example: opt/commvault2/Base

3. Execute the following command to perform an unplanned production failover to the SQL client in a standby CommServe host:

   ./CvFailover -OpType Failover -FailoverType Production -TargetNode StandbySQLClient

   Syntax:

   ./CvFailover -OpType <OpType> -FailoverType <Failover Type> -TargetNode <Target Node Name>

   Where:

   • <OpType> can be one of the following operations:

     Failover - To perform a failover operation.

     GetFailoverConfig - To display the failover configuration status.

     ResetFailoverOperation - To reset a partial failover operation.

     ResetFailoverConfig - To reset an existing failover configuration. (To be used in case of xml corruption.)

   • <Failover Type> can be one of the following failover types:

     Production - To perform a production failover.

     ProductionMaintenance - To perform a maintenance failover.

     MaintenanceFailback option - To reset maintenance failover.

     Test - To perform a test failover.

     TestFailback - To reset a test failover.

   • <Target Node Name> is the name of the target SQL client in the CommServe host to which the operation must be failed over.

   • -forceUnplannedFailover can be used when the setup is stuck and only when the target node is the node with latest database, to avoid major data loss.

4. The following prompt will be displayed:

   # Active node is not reachable, do you want to perform unplanned failover ? [Yes | No]

   Type Yes and press Enter to continue.

Result

• The failover is initiated and the sequence of tasks performed during the process is displayed.

• A DDB Resynchronization job will also be automatically initiated after the failover.

What to Do Next

• After a successful failover, make sure to open the CommCell Console and/or Command Center from the new active CommServe host. Make sure to connect to the currently active host in the CommCell box displayed in the Connect to CommCell dialog box.

  

• If you have failed over to a CommServe host located in a Cloud, or behind a firewall where port 8401 (GxEvMgrS port) is not reachable, make sure to enable the Connect through firewall option, click the Configure button, and then specify the proxy computer details in the Proxy hostname or IP address and Proxy port number boxes. Ideally this proxy computer must be a standalone computer which is accessible to both the CommServe host and the clients.

• If client certificate authentication during installation was disabled before the failover, set the Force per-client certificate authentication option to Yes from the CommCell Console > Control Panel > Certificate Administration dialog box. For more information on enabling this option, see Enabling Client Certificate Authentication on the CommServe Computer.

• If necessary, suspend all schedules after an unplanned failover so that jobs are not started right away.

• After an unplanned failover, ensure that the original CommServe is setup as the standby, as follows:

  • If the original production CommServe host can be restarted, restart the CommServe and verify that this CommServe is set as the Passive node in the Process Manager. For more information about verifying the node, see Verifying the Default Setup.

  • If the original production CommServe host cannot be restarted, setup a new standby CommServe host. For more information about setting up a standby CommServe host, see Reinstalling the Standby CommServe Host.

  • Once the CommServe is setup, verify and ensure that the LiveSync Operation is run from the currently active CommServe host to the standby CommServe host before performing another failover.

• After failover, backups on the new standby CommServe node will be retained based on the retention period. If you want to remove historic data, manually delete the backup job.

Related Topics

• Planned failovers can be performed to alternate between the active and passive CommServe hosts. For more information, see Planned Failovers

• Test failovers can be performed to test the failover process and to verify that the services are successfully started in the standby CommServe host after a failover. For more information, see Verifying Disaster Readiness.

• Maintenance failovers can be performed to install the Commvault Feature Release on the CommServe hosts. For more information, see Installing Feature Releases On CommServe LiveSync Setup.