Loading...

Unplanned Failovers

An unplanned failover can be performed in the following circumstances:

  • When the production CommServe host is inaccessible due to network or hardware failure
  • When the production CommServe host is partially available
  • When there is a failure during a planned failover

Caution: Unplanned failovers can cause data loss, especially if a LiveSync operation is not performed before failing over to the standby CommServe. Use extreme caution while performing unplanned failovers.

Procedure

  1. If the production CommServe host is available, login to the production CommServe host shut down the services associated with the SQL client, in the production CommServe host.
  2. Log on to the standby CommServe host.
  3. Open the Process Manager associated with the SQL client.
  4. On the Failover Assistant tab, select the following options:
    • From the Failover To list, click the name of the passive node that must be used for performing the failover.
    • From the Failover Type list, click Production.
  5. Click Initiate Failover.
  6. On the Confirm prompt, type confirm and click Ok.

Note: For unplanned failovers, it may take a few minutes for the failover process to start, as the process checks whether the SQL nodes are reachable.

Result

  • The failover is initiated and the sequence of tasks performed during the process is displayed in the Process Manager window.
  • If a floating CommServe name is not used, make sure to open the CommCell Console and Command Center from the new active CommServe host, after a successful failover. (Type or choose the name of the active CommServe host in the CommCell list displayed in the Connect to CommCell dialog box.)

    Recommendation: To ensure that the Commcell Console and Command Center points to the correct CommServe, use a floating name. Even if the CommServe is not installed with a floating name, you can create a virtual DNS record and point to the IP address associated with the active CommServe. For other alternatives, see Third-Party Port Mappings (TPPM or Network Proxy).

    If you have failed over to a CommServe host located in a Cloud, or behind a firewall where port 8401 (GxEvMgrS port) is not reachable, make sure to enable the Connect through firewall option, click the Configure button, and then specify the proxy computer details in the Proxy hostname or IP address and Proxy port number boxes. Ideally this proxy computer must be a standalone computer which is accessible to both the CommServe host and the clients.

Post Failover Tasks for a CommServe Using A Floating CommServe Name

Perform the following steps after a failover:

  • Update DNS to re-point the floating CommServe name to the IP address of the standby CommServe host. For information on changing the IP address, refer to the documentation associated with DNS software implemented in your environment.
  • Test and verify that the floating CommServe host resolves to the IP address associated with the standby CommServe host using ping or the equivalent network command.
  • If you have Data Interface Pairs (DIP) defined, make sure that the floating CommServe name is used to connect to the CommServe. This is especially important in older setups where DIPs was configured before setting up the LiveSync feature.

    For more information about setting up DIPs, see Dedicated Backup Network.

  • If clients connect to the CommServe using the two-way Commvault network routes (or network throttle routes), use the following steps to push the network configuration to all clients after a failover.
    1. From the CommCell Browser, open Network Topologies.
    2. Right-click the network topology, and then click Push Network Configuration.
  • If clients connect to the CommServe using one-way network routes from the client to the CommServe, perform the following steps to maintain the floating CommServe Hostname in the DB and registry.
    1. Navigate to the following folder in the active CommServe:

      Program Files\Commvault\ContentStore\Base

    2. Use the following command to set the the floating CommServe Hostname in the DB and registry.

      SetPreImagedNames.exe CSNAME -hostname <CS_Floating_Name> <CS_Physical_name> -skipodbc -passiveCS

      For example:

      SetPreImagedNames.exe CSNAME -hostname floatingCS.company.com passiveCS.company.com -skipodbc -passiveCS

    3. Restart the Commvault Communications Service (GxCVD)on the CommServe. For more information about restarting services, see Restarting a Service on Windows Clients.

Additional Notes

  • If necessary, suspend all schedules after an unplanned failover so that jobs are not started right away.
  • After an unplanned failover, ensure that the original CommServe is setup as the standby, as follows:
    • If the original production CommServe host can be restarted, restart the CommServe and verify that this CommServe is set as the Passive node in the Process Manager. For more information about verifying the node, see the Process Manager section in Verifying the Default Setup.
    • If the original production CommServe host cannot be restarted, setup a new standby CommServe host. For more information about setting up a standby CommServe host, see Reinstalling the Standby CommServe Host.
    • Once the CommServe is setup, verify and ensure that the LiveSync Operation is run from the currently active CommServe host to the standby CommServe host before performing another failover.

Last modified: 12/24/2019 9:40:06 PM