> Expert > CommCell Recovery > CommServe Recovery > CommServe LiveSync For High Availability Disaster Recovery > LiveSync in a Linux Environment > Performing Production Failovers

Unplanned Failovers in a Linux Environment

An unplanned failover can be performed in the event of a disaster, such as the following:

  • When the production CommServe host is inaccessible due to network or hardware failure

  • When the production CommServe host is partially available

  • When there is a failure during a planned failover

Caution

Unplanned failovers can cause data loss, especially if a LiveSync operation is not performed before failing over to the standby CommServe. Use extreme caution while performing unplanned failovers. Use the following steps to perform the failover. Do not manually perform additional steps to initiate the failover.

Procedure

  1. Perform the following steps, if the production CommServe host is available:

    1. Login to the production CommServe host.

    2. If the CommServe is currently in a "lockdown" mode where client certificate authentication is forced during installation, reset the CommServe lockdown by setting the Force per-client certificate authentication option to No from the CommCell Console > Control Panel > Certificate Administration dialog box.

    3. Shut down the services associated with both the CommServe and the SQL client in the production CommServe host.

  2. Open command prompt and navigate to the folder associated with the SQL client:

    For example: opt/commvault2/Base

  3. Execute the following command to perform an unplanned production failover to the SQL client in a standby CommServe host:

    ./CvFailover -OpType Failover -FailoverType Production -TargetNode StandbySQLClient

    Syntax:

    ./CvFailover -OpType <OpType> -FailoverType <Failover Type> -TargetNode <Target Node Name>

    Where:

    • <OpType> can be one of the following operations:

      Failover - To perform a failover operation.

      GetFailoverConfig - To display the failover configuration status.

      ResetFailoverOperation - To reset a partial failover operation.

      ResetFailoverConfig - To reset an existing failover configuration. (To be used in case of xml corruption.)

    • <Failover Type> can be one of the following failover types:

      Production - To perform a production failover.

      ProductionMaintenance - To perform a maintenance failover.

      MaintenanceFailback option - To reset maintenance failover.

      Test - To perform a test failover.

      TestFailback - To reset a test failover.

    • <Target Node Name> is the name of the target SQL client in the CommServe host to which the operation must be failed over.

    • -forceUnplannedFailover can be used when the setup is stuck and only when the target node is the node with latest database, to avoid major data loss.

  4. The following prompt will be displayed:

    # Active node is not reachable, do you want to perform unplanned failover ? [Yes | No]

    Type Yes and press Enter to continue.

Result

  • The failover is initiated and the sequence of tasks performed during the process is displayed.

  • A DDB Resynchronization job will also be automatically initiated after the failover.

Caution

If there is a failure during an unplanned failover, perform one of the following:

  • Perform a Forced Unplanned Failover. Use caution while using this option as you can experience data loss.

  • Contact Commvault Customer Support if you do not want to perform a Forced Unplanned Failover. In this situation, do not perform any manual or additional steps to recover from the failure.

What to Do Next

  • After a successful failover, make sure to open the CommCell Console and/or Command Center from the new active CommServe host. Make sure to connect to the currently active host in the CommCell box displayed in the Connect to CommCell dialog box.

    Post Failover Tasks (1)

  • If you have failed over to a CommServe host located in a Cloud, or behind a firewall where port 8401 (GxEvMgrS port) is not reachable, make sure to enable the Connect through firewall option, click the Configure button, and then specify the proxy computer details in the Proxy hostname or IP address and Proxy port number boxes. Ideally this proxy computer must be a standalone computer which is accessible to both the CommServe host and the clients.

  • If client certificate authentication during installation was disabled before the failover, set the Force per-client certificate authentication option to Yes from the CommCell Console > Control Panel > Certificate Administration dialog box.

  • If necessary, suspend all schedules after an unplanned failover so that jobs are not started right away.

  • After an unplanned failover, ensure that the original CommServe is setup as the standby, as follows:

    • If the original production CommServe host can be restarted, restart the CommServe and verify that this CommServe is set as the Passive node in the Process Manager. For more information about verifying the node, see Verifying the Default Setup.

    • If the original production CommServe host cannot be restarted, setup a new standby CommServe host. For more information about setting up a standby CommServe host, see Reinstalling the Standby CommServe Host.

    • Once the CommServe is setup, verify and ensure that the LiveSync Operation is run from the currently active CommServe host to the standby CommServe host before performing another failover.

  • After failover, backups on the new standby CommServe node will be retained based on the retention period. If you want to remove historic data, manually delete the backup job.

Loading...