DB2 Agent: Data Aging

Getting Started

Data Aging is the process of removing old data from secondary storage to allow the associated media to be reused for future backups.

You can change the retention of your data based on your needs.

For the agent, the retention cycles are calculated both on the subclient as well as on the instance. Therefore, the backup jobs will not age until the data retention cycle for the instance and the data retention cycle for the cycle and subclient are met.

Setting Up the Basic Retention Rule

1. From the CommCell Browser, expand Policies > Storage Policies > storage_policy.

2. Right-click the appropriate storage policy copy, and then click the Properties.

3. In the Copy Properties dialog box, on the Retention tab, under Basic Retention Rule for All Backups, click the Retain For.

   • Enter number of days to retain the data.

   • Enter number of cycles to retain the data.

4. Click OK.

5. In the Confirm Basic Retention dialog box, click Yes.

6. On the ribbon in the CommCell Console, on the Reports tab, click Forecast, and then click Run.

7. The Data Retention Forecast and Compliance report displays the data to be pruned when a data aging job is run.

Running the Data Aging Job

1. From the CommCell Console, right click the CommServe node, click All Tasks > Data Aging.

2. In the Data Aging Options dialog box, in the Job Initiation area, define whether the data aging job runs immediately or if it will be scheduled.

3. Click OK.

   If you chose to run the job immediately, the data aging job starts now.

   If you chose to run the job according to a schedule, the data aging job runs according to the schedule that you defined.

   After data aging job is run, the data will be pruned from the storage.

Retention Rules for Data Only Backups

If a data backup job does not have a linked log backup, it will be retained based on the retention days only and does not follow the retention cycles. However, if the data backup job is linked to a log backup job, the retention will be based on both the number of days as well as the retention cycles.

Retention Rules for Command Line Full Backups

DB2 command line full backup and its associated backups (incremental, differential and log) are pruned based on the cycles.

Incremental backups performed through the CommCell Console after the command line full backup are kept, but will be orphaned when the command line full backup is aged. The DB2 history file has a record of the incremental backups performed through the CommCell Console which depend on the command line full backup, but the database is not recoverable from the incremental backup.

Command line jobs and incremental backups performed through the CommCell Console must not be combined on the same database because they will not be recoverable.

Extended Retention Rules

Extended retention rules allow you to keep specific full backups for an additional period. You can define three additional "extended" retention periods for full backups. For example:

• You may want to retain your weekly full backups for 30 days.

• You may want to retain your monthly full backup for 90 days.

• You may want to retain your yearly full backup for 365 days.

A backup job is selected for extended retention based on its start time. For example: If a backup job starts at 11:55 pm on August 31st and ends at 1 am on September 1st, then it is selected as the last full backup for the month of August and is picked up for extended retention.

In all other cases, we recommend you use Auxiliary Copy for extended storage since it creates another physical copy of the data, thereby reducing the risk of data loss due to media failure.

Note the following regarding extended retention rules:

• All online and offline data-only full backups made from the CommCell Console on a DB2 database that is version 9.5 or above are eligible for extended retention rules.

• Command line online and offline backups are eligible for extended retention only after the subsequent log backup is linked to the command line data backup job.

• You can change the default retention behavior for command line offline full backups by adding the offlineBkp=1 option to the VENDOROPT parameter. This allows command line offline full backups to be also picked for extended retention. Update the DB2 VENDOROPT configuration parameter as follows:

   
  db2 update db cfg for db_name using VENDOROPT "'CvClientName=client_name_on_Commcell,CvInstanceName=commvault_instance_name,offlineBkp=1'"

  This parameter change affects both DB2 command line online and offline full backups. Consequently, you should only run command line offline full backups on the database after the database is configured with the offlineBkp=1 VENDOROPT parameter. If a command line online backup is run it could mistakenly be used for selective copy or extended retention, even if it isn’t recoverable, and will show this extended retention information on the job history after a data aging job runs.

• During a full backup, if the data and logs use the same storage policy, the extended retention rules apply to both the data and the logs.

• Extended retention rules do not apply for log-only backups. However, if the log backup job is linked to a data backup job and uses the same storage policy used by data backups, the extended retention rules of the data backup job are also applied to the linked log backup job.

• If the data and linked logs use different storage policies, the extended retention rules are applied to the data backups only.

Setting Up the Extended Retention Rules

Use the following steps for setting up the extended retention rules:

1. Right-click the storage policy copy and click Properties.

2. Click the Retention tab.

3. Set the basic retention rules by clicking Retain for and entering the number of days and cycles appropriate for your organization.

4. Set the extended retention rules as follows:

   1. Click the For button.

   2. Enter the number of Days Total to retain the backup.

   3. Click the Keep drop-down list, and select the desired backup criteria (for example, Monthly Full, Weekly Full).

   4. Click the Grace Days drop-down list and select the number of days (for example, 2).

      This allows you to consider the additional number of days along with the Extended Retention rule. For example, if the last full backup job fails with in the defined extended retention criteria, then the next full backup job that ran in the specified grace days will be selected for retention.

5. Repeat Step 4 to configure additional extended retention.

6. Click OK.

   

Data Aging for Transaction, Archive, and Logical Log Backups

Log Backups (transaction, archive, or logical logs) are not considered part of the backup cycle. Therefore, storage policy cycle retention parameters do not apply to them. However, log backups may be linked to data backup operations, which can affect their retention as follows:

• Log backups are linked to a full backup if they are run at the same time. This is regardless of whether the full backup included data only or data and logs. Such backups follow the standard data aging rules.

• If a full backup job is run on data, then the next log backup job will be linked to this full backup job.

  These are considered as linked or chained log backups and are not aged until the linked data is aged. In addition, the following is also considered:

  • Logs that need to be copied to secondary copies will not be aged both on primary and non-primary source copy

  • Logs that exist only on one copy will be aged when they are older than the oldest data

  • Logs that exist on multiple copies will be aged according to copy retention days

  • Logs that exist on multiple copies with the longest retention days will be aged when they are older than the oldest data

  • Partial, disabled logs will be aged when they are older than the oldest data

• If a full backup job is run on data and logs, then the next log backup will not be linked to this full backup job.

  As this is an unlinked log backup, by default, this will follow the unique data aging rules for log backups. If you want such log backups to be aged according to the defined days retention rule for the data, you can do so as follows:

  1. From the CommCell Console, click the Storage tab.

  2. Click the Media Management icon.

  3. Click the Data Aging tab.

  4. In the Prune All Database Agent Logs Only By Days Retention Rule, type 1 to enable pruning of all database agent logs based on the days retention.

  5. Click OK.

     

Data Aging Rules for Command Line Backups

• The third party command line log backups can be linked to third party command line data backups as well as any other kind of backup data as per regular data link rule.

• Data from third-party command line backups is aged differently than data from backups initiated through the CommCell Console. Retention cycles are not used for copies involved in operations from the third-party command line. For such operations, data is aged according to the associated retention time. However, you can manually set the retention time for each third party command line job from the storage policy copy. The command line log backups will be aged according to the retention time set for its associated command line data backup job.

• The qoperation agedata command can age data and logs simultaneously based on the Job ID, and it is especially useful for aging each of these items separately.

• Command line full jobs will be aged only when all of the incremental data backups in that cycle are eligible for pruning.

Data Aging Rules for Jobs Completed with Errors

A DB2 backup job is marked as completed with error even if it fails in the log phase. Jobs that are completed with errors are treated as a valid full backup jobs. They are pruned based on basic days retention rules and ignore cycle retention rules. However, both extended retention rules and selective copy will exclude these completed with error jobs.

Data Aging Rules for Selective Copy

A selective copy allows you to copy specific full backup jobs from a source copy. The source copy can be either a primary or a synchronous copy. Selective copy facilitates better tape rotation. Consider the following for DB2:

• By default, a DB2 full backup is copied to a selective copy only if a log backup job also runs and the job is completed after the data backup job. If this is not the case, the data backup job is not copied to the selective copy and will have a "Log Link Pending" status. However, the CommCell Console offline full backups can be copied to a selective copy without a log backup.

  You can change the default behavior by adding the offlineBkp=1 option to the VENDOROPT parameter. This allows command line offline backups to be also picked for selective copy for extended retention. Update the DB2 VENDOROPT configuration parameter as follows:

  db2 update db cfg for <db_name> using VENDOROPT "'CvClientName=<client_name_on_Commcell>,CvInstanceName=<commvault_instance_name>,offlineBkp=1'"

  Only run offline backups on the database after the database is configured with the offlineBkp=1 VENDOROPT parameter. If an online backup is run it could mistakenly be used for a selective copy, even if it isn’t recoverable.

• You can use the sALLOWONLINESELCOPY additional setting and set the value as Y to allow a selective copy of an online full backup that does not include the archive logs.

• A full backup has a one hour delay before it is copied to the selective copy.

• Log jobs are not copied if they are using a different storage policy. The log jobs will be retained by the log policy rules.

• Selective copy of online data-only full backups performed from the CommCell Console is only eligible for version 9.5 or later DB2 databases.

• If data and log backups use the same storage policy, then the linked logs of the full data backup are copied to the selective copy.

For more information, see Selective Copy Overview.

Pruning a Backup Image

You can enable the DB2 triggered pruning of backup images by using the sCvDB2EnablePruning Additional Setting.

1. Set the sCvDB2EnablePruning Additional Setting to y in the registry under <Commvault Instance>/Db2Agent

   Example:

   Instance001/Db2Agent

2. Run the following command on a DB2 Client to prune the backup image:

    
   $ db2 prune history timestamp with force option and delete

   Example:

   $ db2 prune history 20130801192856 with force option and delete
    DB20000I The PRUNE command completed successfully.

When you prune a backup image which includes data and log files, the backup history shows this backup job until it meets the retention policy. In such a case, you must restore only the log files during the Restore To Disk job.

For more information, see Data Aging - Advanced.