Performing Parallel Operating System Upgrades

Operating system (OS) updates can be installed in parallel on horizontally scaled storage pools that have multiple blocks. Parallel update installation process keeps the turn around time needed to perform OS upgrades to the minimum.

Updates can be installed in parallel on storage pools with a minimum of 2 blocks with 6 nodes.
Parallel update process is supported on storage pools with internet access. (Storage pools or nodes without internet access is not supported.)
Parallel updates must be performed using a system console, like KVM.

Procedure

Identify the node with the remote cache.
1. From the navigation pane, go to Manage > System > Maintenance.
  
  The Maintenance page appears.
2. Click Download/copy software.
  
  The Download/copy software section appears, showing the current feature release, the latest available feature release, and the last job that was run.
3. Click Schedule Job .
  
  The Download/copy software dialog box appears.
  
  In Remote caches to sync, expand Servers and make sure that one of the nodes in the cluster is listed. (This is automatically created when you configure the storage pool.)
Login to the node that has the remote cache configured.

Note

Use the system console to login to the node. For example, using KVM, not SSH.
Navigate to the following folder:
```
# cd /opt/commvault/MediaAgent
```
Run the following command to enable parallel upgrade:
```
#./cvupgradeos.py -enable_parallel_upgrade
```
Output similar to the following should be displayed:
```
Successfully enabled parallel upgrade.
```
Using the CommCell Console, download the latest updates and populate the remote cache.
1. From the navigation pane, go to Manage > System > Maintenance.
  
  The Maintenance page appears.
2. Click Download/copy software.
  
  The Download/copy software section appears, showing the current feature release, the latest available feature release, and the last job that was run.
3. Click Run job .
  
  The Download/copy software dialog box appears.
4. Select Download software using Internet or Local Copy, depending on your environment.
5. In Download installation media, select the available feature release or maintenance release installation files that you want to download.
6. In Operating system options, select the Unix check box, and then select Linux x86_64 from the drop-down list.
  
  Note
  
  The latest OS updates will be downloaded, irrespective of the selected option.
7. In Remote caches to sync, select the HyperScale node in which the remote cache is configured.
8. Click Download.
  
  Monitor the progress of the download job to ensure that the software is successfully downloaded and synchronized. For more information, see Viewing Jobs.
  
  Note
  
  It is important to download and populate the remote cache after enabling the parallel upgrade. (Do not skip this step even if you downloaded the updates just before enabling the upgrade.)

Using the system console, run the following command to start the upgrade:

#./cvupgradeos.py

Output similar to the following should be displayed:

...
...
All remote nodes are reachable.
INFO           :
INFO           :        You are attempting to perform OS upgrade of HyperScale nodes.
INFO           :        This will bring down gluster services/volume and reboot the nodes one after the other. Each configured block will be upgraded in parallel, while the nodes in the block will be upgraded sequentially.  All non-gluster nodes will be upgraded in parallel.
INFO           :        Following nodes will be upgraded.
INFO           :                        mynode2.company.com
INFO           :                        mynode3.company.com
INFO           :                        mynode4.company.com
INFO           :                        mynode5.company.com
INFO           :                        mynode6.company.com
INFO           :                        mynode1.company.com
Please confirm if you want to upgrade at this time (y/n):

Verify the nodes, type y, and then press <Enter> to continue.

The upgrade process will be started.

What to Do Next

Once the nodes are upgraded, login to the the node that has the remote cache configured, and verify the status of the upgrade in the following log file:

/var/log/commvault/Log_Files/cvmanager.log

Output similar to the following will be logged after a successful upgrade:

INFO    ================================
INFO             Upgrade Summary
INFO     ================================
INFO     [Block - 28439b5ca011d172bd2574af4655ed2b]
INFO       mynode4.company.com - Successful
INFO       mynode5.company.com - Successful
INFO       mynode6.company.com - Successful
INFO     [Block - a182d125af36655a941793421e5367b5]
INFO       mynode1.company.com - Successful
INFO       mynode2.company.com - Successful
INFO       mynode3.company.com - Successful
INFO     ################### [Main-> Upgrade]: Process Status. ###################
INFO     [Main-> Upgrade]-[pre_process]: Completed successfully.
INFO     [Main-> Upgrade]-[main_process]: Completed successfully.
INFO     [Main-> Upgrade]-[post_process]: Completed successfully.
INFO     [Main-> Upgrade]: Has successfully completed all phases!
INFO     #################################### End #####################################

Additional Information

Check the following log files to troubleshoot errors:
- /var/log/commvault/Log_Files/cvmanager.log
- /var/log/commvault/Log_Files/cvupgradeos.log
If necessary you can disable parallel upgrade using the following command:
```
#./cvupgradeos.py -disable_parallel_upgrade
```
Parallel upgrade process cannot be used to upgrade out of pool nodes using the set_additional_nodes parameter. This parameter, if set, will be ignored and only the nodes in the storage pool will be included.