SAP for Oracle: Data Aging

The Data Aging process performs the following 2 processes.

Aging - marks the jobs that have exceeded retention as aged.
Pruning - physically deletes eligible disk jobs, or recycles a tape when all jobs on it have been marked as aged.

When the data aging job runs, the data aging process compares the current storage policy copy retention settings to the jobs on storage. Any eligible job is marked as aged. The data that has aged can be pruned, depending on the jobs that are on disk or tape.

For the agent, the retention cycles are calculated both on the subclient as well as on the instance. Therefore, the backup jobs will not age until the data retention cycle for the instance and the data retention cycle for the cycle and subclient are met.

For jobs that are on disk:
- If the Managed Disk Space option is disabled, and there are no auxiliary copies dependent on the jobs, the jobs are pruned. This physically deletes the data from the disk.
- If the Managed Disk Space option is enabled, the jobs remain until the disk library reaches the upper watermark threshold defined in the Library Properties.
For jobs that are on tape media:

When all jobs on the tape are marked as aged, and there are no auxiliary copies dependent on the jobs, the tape is moved into a scratch pool and the data is overwritten when the tape is picked for new data protection operation. In this scenario, the data is not deleted and can still be recovered by browsing for aged data, until the tape label is overwritten.

Data Retention Rules

Data Retention rules can be defined as:

Basic Retention Rules
Extended Retention Rules

Basic Retention Rules

You can change the retention rules on a storage policy copy to a desired retention period.

Consider the following before setting up the retention rules:

Configure synchronous copies with a retention period that is greater than or equal to that of the primary copy.

For more information on configuring basic retention rules, see Setting Up the Basic Retention Rules.

Extended Retention Rules

Keep specific full (or synthetic full) backups for an additional period of time by using the extended retention rules.

Extended retention rules can be used in the following circumstances:

If you have a single drive tape library
If you create a hierarchical retention scheme (grandfather-father-son tape rotation)

Define three additional "extended" retention periods for full (or synthetic full) backups by using the extended retention rules. For example:

You may want to retain your weekly full backups for 30 days.
You may want to retain your monthly full backup for 90 days.
You may want to retain your yearly full backup for 365 days.

In all other cases, use the auxiliary copy for extended storage as it creates another physical copy of the data, reducing the risk of data loss due to media failure.

For more information on configuring extended retention rules, see Configuring Extended Retention Rules for instructions.

Enabling Automatic Eligibility of Full Backups for Selective Copy and Extended Retention Rules

When you install Commvault Version 11, all consistent and self-contained full backups are automatically eligible for selective copy and extended retention rules. You can change this option by modifying the Honor Extended Retention Rules for Oracle/RAC/SAP regular Full Jobs Media Management parameter.

When you upgrade to Commvault Version 11, you must set the Honor Extended Retention Rules for Oracle/RAC/SAP regular Full Jobs Media Management parameter to 1 to have all consistent and self-contained full backups to be automatically eligible for selective copy and extended retention rules.

SAP Oracle selective online full backup jobs from the CommCell Console and command line are also eligible for selective copy and extended retention rules.

For information on configuring the Media Management parameter, see Media Management Configuration.

When Log Backups Age with Full Backups

Log Backups (archived redo or logical) are not considered part of the backup cycle. Therefore, the storage policy cycle retention parameters do not apply to them. However, log backups can be linked to data backup operations, which can affect their retention as follows:

When a log backup runs at the same time as a full backup, they are linked even when the full backup included data only. These backups follow the standard data aging rules.
If a full backup job includes data and logs, then the next log backup is not linked to this full backup job. These are unlinked log backups and by default and follow these log backup data aging rules:
- Logs that are copied to secondary copies are not aged on both the primary and non-primary source copy.
- Logs that exist on only one copy are aged when they are older than the oldest data.
- Logs that exist on multiple copies are retained on the copy with longest retention days and are aged after the oldest data. The log jobs on the remaining copies are aged according to the copy retention days without checking if the oldest data exists.
- Partial, disabled logs are aged when they are older than the oldest data.
If a full backup job includes just the data then the next log backup job is linked to this full backup job. These are considered as linked or chained log backups and are not aged until the linked data is aged. In addition, these log backups follow the unique data aging rules for log backups.

Disabling the Log Retention Rule (Pruning All Log Backups by Days)

When you disable the log retention rule, unlinked log backups age according to the defined days retention rule for the data. When you define the days retention rule, use the same storage policy or retention criteria used for traditional RMAN backups, backup copy, and traditional RMAN log backup operations.

For example, consider the following subclients:

A clone subclient with a clone storage policy
A IntelliSnap subclient with a IntelliSnap storage policy
A log subclient with a log storage policy

The clone jobs using backup copy are run daily, IntelliSnap jobs without backup copy are run every hour, and regular log backup jobs are run every 15 minutes.

When you disable the log rule, make the retention for the log storage policy primary copy the same or more than the retention set for the clone storage policy primary copy. Alternatively, you can assign the same clone storage policy for archive log backups.

From the CommCell Console ribbon, click the Home tab, and then click Control Panel.
Under the Storage section, click the Media Management icon.
On the Data Aging tab of the Media Management Configuration dialog box, in the Prune All Database Agent Logs Only By Days Retention Rule, type 1 to enable pruning for all database agent logs based on the days retention.
Click OK to close the Media Management Configuration dialog box.

Selective Online Full Backup Retention Rules

A selective online full operation includes archive logs and data. Any log job you start within the time frame of the selective online full operation is linked to the selective online full job. These logs and the selective online full are then considered as one entity within the software. They are copied to synchronous and selective copies together during auxiliary copy operations and are aged together. If any part of the selective online full is missing from a copy, the full backup is not considered as a valid full and in not counted as a cycle during data aging. Consider the following:

The data from selective online full backups are considered the same as offline full backups for each Oracle subclient in terms of extended retention rules of days. Selective online full backups and all logs linked with it must be retained together on the same storage policy copy.
The logs that are linked with a selective online full (and the logs of the selective online full) can be aged only if they are older than the oldest data that can be aged and the corresponding data of the selective online full that can be or have been aged.
Selective online full backup jobs that are completed with errors are not retained by extended retention rules during the data aging operation.

Command Line Retention Rules

You can link command line log backups to third-party data backups, as well as any other kind of backup data as per the regular data link rule.

Retention cycles are not used for copies involved in command line operation. For such operations, data is aged according to the associated retention time. You can manually set the retention time for each command line job from the storage policy copy. The command line log backups are aged according to the retention time set for its associated command line data backup job.

You can run selective online full backups from the CommCell Console. Third party command line selective online full backups that use the util_file or util_file_online interfaces, are copied to a selective copy and only the basic retention rules apply. Only CommCell Console selective online full backups are picked for extended retention. The data forecast and compliance report displays “REQ_BY_DATA” and “LOG RULE” for log backups retained that are retained as part of a third party command line selective online full backup.

Note: Command line jobs honor the day retention, but do not honor the cycle retention.

Data Aging Rules for Jobs Completed with Errors

Cycle retention is not honored on jobs that complete with errors. The job gets pruned after the configured storage policy retention (the Basic Retention Rule for All Backups on the Retention tab of the Storage Policy Properties dialog) expires.

From the CommCell Console ribbon, click the Home tab, and then click Control Panel.
Under the Storage section, click the Media Management icon.
On the Data Aging tab of the Media Management Configuration dialog box, change the value for the Ignore Completed With Errors job option for Extended Retention calculations option from 1 to 0.
Click OK to close the Media Management Configuration dialog box.

Data Aging Rules for Jobs Completed with Warnings

For jobs that are completed with warning are treated as regular full backup jobs and utilize the same data aging rules defined for regular full jobs.