On-Demand Data Protection Operations

Overview

On-Demand Data Protection Operations allow content to be specified as an external input at the time of initiating a data protection operation. Whereas traditional backups/archive operations are performed on subclients, which have fixed content configured prior to performing the operation, On-Demand Data Protection Operations allow you the flexibility of specifying content each time you perform a backup or archive operation. Although the concept is the same, the implementation differs somewhat based on the type of agent, as described below.

Windows/Unix/Macintosh File System Agents

Content for on-demand backups is specified through the use of a Directive File and one or more Content Files, which back up data for a subclient of an On-Demand Backup Set. For more information, see Defining Content for On-Demand Data Protection Operations.
NDMP Agents

Content for on-demand backups is specified directly through the use of a Content File (no Directive File is required), which backs up data for a subclient of an On-Demand Backup Set. For more information, see Content File.
File Archiver for Windows

Content for on-demand data protection operations can be defined through either of the following methods:
- On-Demand File List, which can work along with existing subclients and rule processing to archive data that you specify. See On-Demand File List for more information.
- On-Demand Archive Set, which uses a specialized subclient dedicated to this purpose and does not perform rule processing. See Defining Content for On-Demand Data Protection Operations for more information.
Oracle Agent

Content for on-demand backups is specified through RMAN scripts submitted through the Command Line Interface using an On-Demand Instance. See On-Demand Instances and Running RMAN Scripts using the Command Line Interface for more information.
Microsoft SharePoint Server Agent

Content for On-Demand Backup Set for Database is specified through the use of a Directive File. Directive File should contain only site collection's url. For more information, see Directive File.
Microsoft SQL Server Agent

On-Demand backups allow content to be specified as an external input at the time of initiating a data protection operation. For more information, see Configuring SQL Server Agent Using the Command Line Interface.

Defining Content for On-Demand Data Protection Operations

Content for On-Demand Data Protection operations is defined by the user in the form of text files called the Content File. In addition, some iDataAgents utilize a Directive File to list the location of the Content Files. It is important that you understand the purpose of each of these files, and what they contain in order to successfully utilize the On-Demand Data Protection feature. Details on these files are provided below.

Content File

The Content File is a text file that you create for the purpose of telling the system where the data is located that you want protected on demand. Each Content File contains the fully qualified paths from the root directory to files, links and/or devices to be backed up/archived. For example:

Sample Content File on Windows

Sample Content File on Unix/Macintosh

Sample Content File for NAS File Server

c:\data\datafile.txt
c:\data\wordfile.doc
d:\my documents\webdoc.html

UNC Path
\\client1\shares\ondemand_content\test1.txt
\\client1\shares\ondemand_content\test2.txt
\\client1\shares\ondemand_content\test3.txt

/usr/datafile
/usr/textfile
/etc/docfile

NetApp
/vol/vol0/Dir1
/vol/vol0/Dir2
/vol/vol1/Dir1

Celerra
/fs1/Dir1
/fs1/Dir2
/fs1/Dir3

You are allowed to use multiple Content Files for an On-Demand Data Protection operation, provided that they are listed in the Directive File. For NDMP Agents and the File Archiver for Windows Agent, which do not use a Directive File, only one Content File is used.

Content File Location

For NDMP Agents, the Content File must be placed on the CommServe or the MediaAgent; in all other cases, the Content File must be placed on the client.

Creating the Content File

To create the Content File, set up a text file that contains the fully qualified paths from the root directory to the data you want backed up/archived. These paths can include files as well as entire directories. If you include directories, the system will automatically expand all subdirectories within the specified directory and back up/archive all the data within them as well.

NOTES

Wildcards or regular expressions are not supported in the Content File.
Unless you have set the OnDemand_AutoExpandDir additional setting to turn off the automatic expansion of nested directories, overlapping content, illustrated in the following example as the webfiles directory, will be backed up or archived twice:
c:\data\files\
c:\data\files\webfiles\
For NDMP Agents, list directories as content, not individual files.
For NDMP Agents and the File Archiver for Windows Agent, the Content File is the only file you need to create; a Directive File as described below is not used. When performing an On-Demand Data Protection operation, the Content File is used directly instead of the Directive File. Refer to Run an On-Demand Data Protection Operation.

Adding Files and Folders with Unicode Characters

If the path or the filename contains Unicode characters, the Content File must be converted to a format that can be used by the data protection operation. This can be done using the CVconvertUnicode utility. For a detailed explanation on how to use this tool, see Unicode Conversion Utility.

Directive File

The Directive File is a text file that you create on the client for the purpose of the telling the system where the Content Files are located in order to perform an On-Demand Data Protection operation. (A Directive File is not required for the NDMP Agents or File Archiver for Windows Agent.) The Directive File contains the fully qualified paths from the root directory to one or more Content Files. For example:

Sample Directive File on Windows	Sample Directive File on Unix/Macintosh	Sample Directive File on SharePoint
c:\temp\ContentFile1.txt c:\temp\ContentFile2.txt d:\ContentFile3.txt	/usr/ContentFile1 /usr/ContentFile2 /etc/ContentFile3	http://med/sites/itdeveloper http://med/sites/newproduct

Each On-Demand Data Protection operation requires exactly one Directive File. That is, the system will only let you enter one Directive File for the operation.

Creating the Directive File

First create the Content Files. Then create the Directive File by setting up a text file on the client computer containing the fully qualified paths from the root directory to the Contents Files.

NOTES

Blank lines in the Directive File are not supported. Therefore, do not include blank lines in the Directive File when you are creating this file.
Wildcards or regular expressions are not supported in the Directive File.

How to Run an On-Demand Data Protection Operation

The following section provides the steps for using On-Demand Data Protection Operations:

Create a New On-Demand Backup Set/Archive Set.
Create the Content File.
Create the Directive File if your iDataAgent requires one.
Run an On-Demand Data Protection Operation.

Best Practices

Consider the following when using On-Demand Data Protection operations:

Perform an On-Demand Data Protection operation when the contents change dynamically. Perform subclient-based data protection when the contents do not change.
To perform On-Demand Backups/Archives with multiple streams, create multiple Content Files. As each Content File directly translates into a stream, we recommend that each stream is reading off a different disk. However, to ensure that concurrent read operations on the same disk do not cause performance degradation, multiple Content Files should be used with care. For more information, refer to the Notes regarding Streams and Data Readers.

General Information

This section contains information of a general nature regarding On-Demand Data Protection Operations.

On-Demand Backup Sets and Archive Sets

The On-Demand Backup Set and On-Demand Archive Set are logical entities that allow data to be protected on demand, with content specified at the point of the operation. The primary difference between an On-Demand Backup Set/Archive Set and a traditional backup set/archive set, is that data protection operations for the subclient associated with an On-Demand Backup Set/Archive Set only support On-Demand Data Protection operations. Other points to keep in mind are as follows:

On-Demand Backup Sets/Archive Sets have the same set of Property tabs as traditional backup sets/archive sets.
On-Demand Backup Sets/Archive Sets and their default subclients support the same shortcut menu operations as traditional backup sets/archive sets and subclients.
Only one subclient is allowed for an On-Demand Backup Set/Archive Set. For UNIX File System iDataAgents, multiple user-defined subclient creation is supported for an On-Demand Backup Set.
A Legal Hold Set is a special type of On-Demand Backup Set/Archive Set that is automatically created on the CommServe’s File System iDataAgent for each Legal Hold. For more information, see Legal Hold.
A Silo Storage Set is a special type of Backup Set/Archive Set that is automatically created on the File System iDataAgent installed on the CommServe computer for each Silo Storage. For more information, see Deduplication to Tape.

Default Subclients for On-Demand Backup Sets/Archive Sets

An On-Demand Backup Set and On-Demand Archive Set contain a default subclient that functions differently from ordinary default subclients of other backup set/archive set types (i.e., default and user-defined). These differences are listed below.

The content to be backed up/archived is not defined from the Subclient Properties (Contents) tab but is defined in one or more Content Files.
The default subclient of an On-Demand Backup Set/Archive Set does not include data from all fixed disk resources on the client that is not allocated to other subclients. Since the default subclient of an On-Demand Backup Set/Archive Set is not designed to serve as a catch-all mechanism to ensure that all data on the client is protected, it does not include any contents other than what is listed in the user-specified Content Files.
There is no Subclient Properties (Filters) tab, and thus subclient level filtering is not available but global filters enabled at the CommCell level are honored.
For the File Archiver for Windows Agent, the File Rules tab is not available for the default subclient of an On-Demand Archive Set. Since no rules processing is performed for On-Demand Archive operations, this tab is not needed. However, the Stub Rule tab still applies and must be configured on the default subclient after creation of the On-Demand Archive Set in order to tell the system whether to create stubs and how long they should be retained.

On-Demand Data Protection Operations for Oracle iDataAgent

You can perform on demand data protection operations for Oracle iDataAgent, by creating On-Demand Instances.

On-Demand Instance is a logical entity that allows Oracle data to be protected on demand, with content specified in an RMAN script that is run through the Command Line Interface. The primary difference between an On-Demand Instance and a traditional Instance, is that data protection operations for the subclient associated with an On-Demand Instance only supports On-Demand Data Protection operations.

Review the following before creating On-Demand Instances:

The On-Demand Instance must be created in order to run RMAN scripts through the Command Line Interface. See Create an On-Demand Instance for step-by-step instructions.
Supports the ability to view backup history and restore history, as well as delete the On-Demand Instance object.
Supports only Command Line - based backup.
On-Demand Instance uses only the default subclient.
Supports the ability to view the RMAN log for a running Oracle On-Demand Instance job.

The RMAN log files are in the Commvault JobResults directory. For a Windows configuration the default location is:

C:\Program Files\CommVault\ContentStore\iDataAgent\jobResults\CV_JobResults\2\0\<job_id>\backup.out

For a UNIX configuration, the location is:
```
 
/opt/commvault/iDataAgent/jobResults/CV_JobResults/2/0/<job_id>/backup.out
```
Make sure the CommServe and Oracle clients are upgraded to the current version in order to view the RMAN log for running Oracle jobs.
Do not create On-Demand instances to run maintenance scripts. Instead, run third-party command line jobs to perform maintenance operations.

Default Subclients for On-Demand Instances

An On-Demand Instance contains a default subclient that functions differently from ordinary default subclients of other instances. These differences are listed below.

The default Subclient of an On-Demand Instance must exist in order to run RMAN scripts through the Command Line Interface.
The content to be backed up is not defined from the Subclient Properties (Contents) tab, but is defined in an RMAN script and run through the Command Line Interface.
The default subclient of an On-Demand Instance does not include data from all fixed disk resources on the client that is not allocated to other subclients. Since the default subclient of an On-Demand Instance is not designed to serve as a catch-all mechanism to ensure that all data on the client is protected, it does not include any contents other than what is listed in the RMAN script.
Supports property tabs for establishing Pre-processes and Post-Processes, setting the Data Transfer Options, enabling Activity Control and Encryption.
Supports the ability to view backup history and list media.
There is no Subclient Properties (Filters) tab, and thus no automatic filtering is available.
Storage policies for the on demand data protection operation must be defined through the RMAN script.
Supports only Command Line - based backup and hence scheduling a backup is not possible.
Deleting the parent object (for example, On-Demand Instance) will delete the default Subclient.

See Oracle RMAN Parameters for information on the RMAN parameters supported by the On-Demand Instance.

You can run Oracle On-Demand Operations from the Command Line Interface using Save as Scripts. For more information, see Command Line Interface.

Restoring or Recovering Data from On-Demand Data Protection Operations

Data from On-Demand Data Protection operations can be restored through ordinary operation or using restore by jobs operation. See Restore Backup Data, or Restore By Jobs for the respective overview.

Command Line Interface

The Command Line Interface supports the capability to create, rename, delete or back up an On-Demand Backup Set, as well as back up an On-Demand Instance. See Command Line Interface for an overview.

Oracle On-Demand Operations can also be executed using save as scripts. To do this, you will need to save a regular Oracle backup or restore operation as a script (XML) file, and then include the path to the RMAN script in the XML file.

Example:

<scriptItem>/database/oracle10g2/onejob/rman_restore.txt</scriptItem>

You can run the operation by using the qoperation execute command.