You can stop and restart a single node in a HyperScale cluster when required. To maintain data integrity and prevent data loss, always follow the shutdown and startup procedures outlined in this topic. Use these steps to safely shut down and restart a single node for maintenance purposes.
Caution
- If more than one node must be shut down, shut down the entire cluster using the procedure described in Stopping and Starting the Cluster. Improperly shutting down multiple nodes individually can result in data integrity issues, data loss, and restore failures.
- Before stopping a node, verify the cluster status by running the following command to confirm that all services are fully online on the remaining nodes:
After starting a node, it may take up to one hour for all services to come online. Do not stop the next node until you confirm the services are fully operational. Stopping another node prematurely may result in two nodes being offline simultaneously, which can compromise data integrity./usr/local/hedvig/scripts/showmembers.exp
Before You Begin
-
If you have a Primary and Standby CommServe server setup, verify whether the CommServe server is active on the node you want to stop.
-
Log in to the Command Center. For details, see Accessing the Command Center.
-
From the navigation pane, expand Administration > System, and then click CommServe LiveSync.
-
On the CommServe LiveSync page, under Nodes, verify the Node Information to identify the active and passive nodes.
-
-
If the CommServe server is active on the node you want to stop, fail over the CommServe server to the standby node. For instructions, see Planned Failovers.
Procedure
Perform an operation based on your requirement.
Stopping a Node
-
Stop or suspend all running jobs that are associated with the MediaAgent installed in the node that you plan to shut-down. For more information, see Controlling Jobs.
-
Place the MediaAgent in maintenance mode. For details, see Setting the MediaAgent on Maintenance Mode.
-
Stop the Commvault services and verify that the Commvault File System (CVFS) vdisk is unmounted:
-
Using an SSH client (for example, PuTTY), log in to a node with root credentials.
-
Stop the services:
# commvault stop
-
Verify that the vdisk is unmounted:
The output should not display entries similar to:# df -h
127.0.0.1:/exports/CVLTBackupSMC43002153 269T -1.5T 271T 0% /ws/hedvig/CVLTBackupSMC43002153 127.0.0.1:/exports/CVLTBackupSMC43002153 269T -1.5T 271T 0% /ws/hedvig/CVLTBackupSMC43002153-r
-
-
Stop the node:
Note
On nodes with HyperScale 2.x installations, before you stop the nodes, switch to the admin user:
# su - admin
The cluster name appears as shown in the following sample output:# export HV_PUBKEY=1 # /opt/hedvig/bin/hv_deploy --show_all_clusters
Cluster name: HV13102021054000 - owner: unowned - version: v-4.5.0.0.2.3597.00320e0ba9:06aca75bc4db06170a13ef14cbfe8446
Use the cluster name to stop the node:
# /opt/hedvig/bin/hv_deploy --stop_node fqdn hostname --cluster_name Clustername
Example:
# /opt/hedvig/bin/hv_deploy --stop_node hs2300m5.company.com --cluster_name HV13102021054000 ... provisioning as a hsx cluster. details: ApplianceType HS2300 ... logged into commvault cluster Ensuring password-less SSH connections have been added for all cluster machines. RUNNING: stop_node ************************************************************* RUNNING PLAY 1 [stop hedvig hblock] ******************************************** RUNNING PLAY 2 [stop hedvig pages] ********************************************* RUNNING PLAY 3 [stop hedvig hpod] ********************************************** COMPLETED PLAY 3 *************************************************************** ACTION RECAP: ****************************************************************** hs2300m5.company.com : ok=9 changed=0 unreachable=0 failed=0
Starting a Node After a Reboot
-
Reboot the node:
# reboot
After the reboot, services start automatically and the CVFS vdisk is mounted.
-
Disable maintenance mode for the MediaAgent to bring it online. For details, see Setting the MediaAgent on Maintenance Mode.
Starting a Node Manually
-
Using an SSH client, log in to a node with root credentials.
-
Start the node:
Note
On nodes with HyperScale 2.x installations, before you start the nodes, switch to the admin user:
# su - admin
.# export HV_PUBKEY=1 # /opt/hedvig/bin/hv_deploy --show_all_clusters
The cluster name appears as shown in the following sample output:
Cluster name: HV13102021054000 - owner: unowned - version: v-4.5.0.0.2.3597.00320e0ba9:06aca75bc4db06170a13ef14cbfe8446
Use the cluster name to start the node:
# /opt/hedvig/bin/hv_deploy --start_node fqdn hostname --cluster_name Clustername
Example:
# /opt/hedvig/bin/hv_deploy --start_node hs2300m5.company.com --cluster_name HV13102021054000 ... provisioning as a hsx cluster. details: ApplianceType HS2300 ... logged into commvault cluster Ensuring password-less SSH connections have been added for all cluster machines. RUNNING: start_node ************************************************************ RUNNING PLAY 1 [collect deploy server facts] *********************************** RUNNING PLAY 2 [start node] **************************************************** COMPLETED PLAY 2 *************************************************************** ACTION RECAP: ****************************************************************** hs2300m5-1a.company.com : ok=16 changed=10 unreachable=0 failed=0 localhost : ok=2 changed=0 unreachable=0 failed=0
-
Start the Commvault services (this also mounts the CVFS vdisk.)
# commvault start
-
Disable maintenance mode for the MediaAgent to bring it online. For details, see Setting the MediaAgent on Maintenance Mode.