IntelliSnap Backup Failures
Failed in Indexing Operation (Create or Update Index)
Make sure to perform a crosscheck archive log all operation. Then, retry the IntelliSnap backup operation.
Failed during VSS IntelliSnap Backup
Verify the Windows Event Viewer, if any error occurs during the VSS IntelliSnap backup operation.
There is a known issue with Oracle VSS Writer memory leak (metalink ID 1358570.1). If Oravssw.exe uses more memory, restart the service.
IntelliSnap Backup Fails
IntelliSnap Backup operations fail if the database is in the NOARCHIVELOG mode. Alter the database to ARCHIVELOG mode and then perform the IntelliSnap backups.
SQL>startup mount;
SQL>alter database archivelog;
SQL>alter database open;
Sometimes, the IntelliSnap backup operations may fail if you enable the SSKIPBACKUPBROWSE additional setting. Disable it.
To add or edit an additional setting, follow the steps in Add or Modify an Additional Setting.
Mount Operation Fails on Windows
During RMAN backup copy operation on proxy, the mount operations fails with the following error message:
OS mount failed : [Mount failed: E_FAIL (MM.60611)]
To resolve this issue make sure the mount directory is empty and there are no drives (same as source data drives) exist on the proxy.
IntelliSnap Restores of High Transaction Rate Databases May Fail
Do not use IntelliSnap to back up archive log volume of high transaction databases, because the following errors may be encountered while restoring from those snaps or during the backup copy of those IntelliSnap jobs.
ORA-00283: recovery session canceled due to errors
ORA-00354: corrupt redo log block header
ORA-00353: log corruption near block 86016 change 7963079914 time 01/28/2014 09:54:46
Instead back up the archive logs using RMAN. Select the Use Rman for Log Backup check box on the Logs Backup tab of the Subclient Properties dialog box.
IntelliSnap Backup Job Runs Indefinitely
IntelliSnap Backup operations may run indefinitely due to following reasons:
-
If the archive log location is full. In such cases, you have to either clear the archive logs to make enough space available or specify a different archive log location.
-
If there are faulty or failed multipath devices. If faulty devices exist, delete them using the following command on a linux computer.
dmsetup message <faulty multipath device> 0 "fail_if_no_path" multipath -f <faulty multipath device>
Oracle RMAN Snap to Tape Incremental Copy Fails on the Proxy Computer
When performing an Oracle RMAN incremental snap to tape copy, note the following:
-
The Oracle database installed on the proxy machine should be of the same version as the source. For example, if Oracle 10.2.0.4 is installed on source then the proxy also should be of the same version.
-
Oracle user ID/group ID on source and proxy should be the same otherwise the RMAN backup copy will fail with permission issues.
-
Copy the Oracle parameter file pfile from the source to the proxy (say the instance only as spfile)
sqlplus <username/password@servicename> as sysdba << EOF Create pfile from spfile; Exit; EOF
Copy the pfile init<instance name>.ora to the proxy computer and the destination location should be $ORACLE_HOME/dbs/ with oracle user permissions. Also, copy the oracle password file from the source to the proxy computer's $ORACLE_HOME/dbs/ directory.
-
Create the bdump, udump, adump, cdump and diagnostic_dest directories. Please note that the directories should be in the same location as the source.
-
Create the directories DB_CREATE_FILE_DEST, LOG_ARCHIVE_DEST and any other directory required for starting the database in NOMOUNT mode. If there are multiple archive destinations, then create the directories for each of the archivelog destinations.
-
Copy $ORACLE_HOME/network/admin/tnsnames.ora configuration from source to proxy. If the entire content cannot be copied then copy at least the configuration related to catalog connection.
-
Startup the proxy instance in NOMOUNT mode.
-
Now configure the proxy Oracle Instance on the CommServe Console and status should be started. Now you are all set to do Oracle RMAN snap to tape incremental
Note that for incremental backup, snap clone will be mounted in the same location as the source mount-point of the source database. For example, if the data mount-point is /netapp/data then on the proxy too it will be mounted in /netapp/data, similarly for the archive log location chosen for the IntelliSnap backup. Therefore, ensure that on the proxy this mount-point is free and there is no such directory existing on the proxy computer ( even if it exists it should be empty).
IntelliSnap Backup Copy on the Proxy Fails
Sometimes, the Backup copy on proxy fails with the following error: ORA-7217 sltln: environment variable cannot be evaluated.
Example:
If RMAN configuration parameters contain $s as shown below:
RMAN> show all;
RMAN configuration parameters for database with db_unique_name are:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 0 DAYS;
CONFIGURE BACKUP OPTIMIZATION ON;
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP OFF;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '$ORACLE_BKUP/$ORACLE_SID/controlfile%F.f';
CONFIGURE DEVICE TYPE DISK PARALLELISM 2 BACKUP TYPE TO COMPRESSED BACKUPSET;
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '$ORACLE_BKUP/$ORACLE_SID/%d_df_%t_b%s_p%p.rmf' MAXPIECESIZE 2000 M;
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '$ORACLE_BKUP/$ORACLE_SID/snapcf.f';
ORA-7217 sltln: environment variable cannot be evaluated
Perform one of the following steps to resolve this issue:
-
Configure “/” as connect string on proxy and set the environment variables specified for snapshot control file path in cvprofile residing in Base directory.
-
Edit /opt/commvault/Base/cvprofile
-
export ORACLE_BKUP=/tmp
-
commvault restart
-
-
Change the configuration parameters on source to remove environment variables ($ORACLE_BKUP etc).
Sometimes, the backup copy on proxy fails with the following error even though the control file copy is cataloged into recovery catalog:
RMAN-03002: failure of backup command at 04/11/2012 13:44:03
RMAN-06004: ORACLE error from recovery catalog database: RMAN-20220: control file copy not found in the recovery catalog
RMAN-06090: error while looking up control file copy: +DATA/backup.ctl.galaxy
RMAN>
Perform the following to resolve this failure:
-
Unregister and then reregister the source database to recovery catalog.
-
Resume the backup copy job.
Sometimes the IntelliSnap operation and a backup copy on proxy for ASM database fails with the following errors:
For Snap:
Error Code: [19:1335]
Description: Oracle Backup [GetASMLogDisks Failed.]
For Backup Copy:
Error Code: [19:1335]
Description: Oracle Backup [Mounting snap or renaming ASM DiskGroup operation failed with an error. Please check the logs for more details.]
30156 f48b7410 04/18 13:00:37 97550 OraObject::GetOraMode() - oraMode = SHUTDOWN.
30156 f48b7410 04/18 13:00:37 97550 OraObject::GetOraMode() - oraMode = SHUTDOWN: return Error.
30156 f48b7410 04/18 13:00:37 97550 OraInfoBase::GetInfo() - CheckOraMode() failed: oraError=301989906
30156 f48b7410 04/18 13:00:37 97550 ASMSnapUtil::runSqlWithScript() - Failed while getting the Oracle version
30156 f48b7410 04/18 13:00:37 97550 ASMSnapUtil::runSqlWithScript() - Writing into file [@/opt/commvault/Base/Temp/tmp_asm_30156.sql] sql = [select 'U,'|| state from v$asm_diskgroup where name = 'DATADG1'
/
]
30156 f48b7410 04/18 13:00:37 97550 ASMSnapUtil::runSqlWithScript() - Executing SQL select 'U,'|| state from v$asm_diskgroup where name = 'DATADG1'
/
failed with an error Database is in SHUTDOWN mode
30156 f48b7410 04/18 13:00:37 97550 ASMSnapUtil::isASMDiskGroupMounted() - Failed while executing the sqlscript [select 'U,'|| state from v$asm_diskgroup where name = 'DATADG1'
/
] output = []
30156 f48b7410 04/18 13:00:37 97550 ASMDiskGroup::renameASMDiskGroup() - Child change user=oracle, gid=501, uid=501
30156 f48b7410 04/18 13:00:37 97550 ClOraSnapAgent::RenameAndMountASMDiskGroup() - Successfully Renamed ASM DISK GROUPS
30156 f48b7410 04/18 13:00:37 97550 OraObject::GetOraMode() - strictSID = 0
30156 f48b7410 04/18 13:00:37 97550 OraChildProcess::SetPostForkParam() - Parent path = /oracle11gr2/product/11.2.0/dbhome_1/bin/sqlplus
30156 f48b7410 04/18 13:00:37 97550 OraChildProcess::SetPostForkParam() - Parent oraUser = oracle
Perform the following to resolve this failure:
-
Log in to the CommCell Console.
-
Verify the Oracle +ASM instance status. The status should be in started mode.
-
Resume the IntelliSnap operation or the backup copy job that you need to perform.
Latency Occurs for Releasing File Descriptors by an ASM Instance during the Dismount of ASM Diskgroups
On Oracle version 11.2.02, there is a latency for releasing file descriptors by ASM instance during dismounting of ASM diskgroups.
Apply the following patch on Oracle version 11.2.02 to resolve the latency issue:
Patch:11666137
https://updates.oracle.com/download/11666137.html
IntelliSnap Backup Copy of an Oracle ASM Database Fails If You Perform an IntelliSnap Operation without Disabling the Snap Integrity for Persistent Snap Engines
If you perform a IntelliSnap operation without disabling the Snap Integrity for persistent snap engines, the following error is displayed:
19935 b7f476d0 03/01 12:20:03 11972 ASMDiskGroup::mountASMDisk() - Failed while excuting sql [alter system set asm_diskstring='/ora_snap*/asm*','/ora_snap*/*'
/
alter diskgroup DATA mount
/
] output = [ SQL*Plus: Release 11.2.0.1.0 Production on Thu Mar 1 12:20:03 2012,Copyright (c) 1982, 2009, Oracle. All rights reserved.,SQL> SQL> SQL> SQL> Connected.,SQL> SQL> SQL> SQL> SQL> SQL> SQL> ,System altered.,alter diskgroup DATA mount,ERROR at line 1:,ORA-15032: not all alterations performed,ORA-15017: diskgroup "DATA" cannot be mounted,ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA",SQL> SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production,With the Automatic Storage Management option,]
19935 b7f476d0 03/01 12:20:03 11972 ASMDiskGroup::mountASMDisk() - One of the diskgroups could not be mounted
19935 b7f476d0 03/01 12:20:03 11972 ClOraSnapAgent::RenameAndMountASMDiskGroup() - mountASMDiskGroup failed
Perform the following steps to resolve this issue:
-
Mount the snaps from the CommCell console (List Snapshots-->Mount) on Proxy and then use the following command:
/opt/commvault/Base/diskgroup_rename.pl -y <original source diskgroup name> <mounted disk path>
For example, if snap is mounted on /tmp and the original diskgroup name is DATA01, then perform the following command:
[oracle@Hindu ~]$ /opt/commvault/Base/diskgroup_rename.pl -y DATA01 /tmp/volume_1335990104/DATA01 renaming disks to: DATA01 current name for /tmp/volume_1335990104/DATA01: DS2995770 new name for /tmp/volume_1335990104/DATA01: DATA01
-
Unmount the Snap. Perform the same for all the snaps in that IntelliSnap operation.
Changing the Snapshot control file and SP file Location to a Snappable Volume
Run the following RMAN command to change the path for the snapshot control file:
Configure Snapshot CONTROLFILE NAME TO '<Snappable volme>\SNCFVSSDB.ora';
Remove the SP file from the $ORACLE_HOME/database location or move sp file to snappable location and add this SPfile location to pfile.
Oracle VSS IntelliSnap Job Fails on Windows
XML Document is Long
Oracle VSS Snap job fails with "failed to finalize shadow" and windows events show the following error message:
XML document is too long.
Oracle stores the redo logs in BCD XML. Hence, the XML file size may be large if there are a lot of transactions. Therefore, rerun the IntelliSnap operation when there is less I/O activity on the oracle database.
Device Not Filer Volume
When performing VSS IntelliSnap job after restoring the control file, the job may fail with one of the following errors:
Device is not filer volume
Device Not Found
This happens because the Oracle VSS writer includes a snapshot control file for RMAN, which by default is in the %ORACLE_HOME%/database location and may not be on a snapable volume.
To resolve this, run the following RMAN script to change the location of the snapshot control file:
RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '<Snappable Volume>\SNAPSHOT.CTL';
Example:
RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO 'F:\SNAPDB_LOG\SNAPTEST\SNAPSHOT.CT L';
Output:
using target database control file instead of recovery catalog
old RMAN configuration parameters: CONFIGURE SNAPSHOT CONTROLFILE NAME TO 'F:\SNAPDB_LOG\SNAPTEST\SNAPSHOT.CTL';
new RMAN configuration parameters: CONFIGURE SNAPSHOT CONTROLFILE NAME TO 'F:\SNAPDB_LOG\SNAPTEST\SNAPSHOT.CTL';
new RMAN configuration parameters are successfully stored
RMAN> exit
Tablespace and Data file Restores from Snap That use the File System Fail on Windows
During partial tablespace/data file restore from snap using File System, if the database is open, the data file restore operation may fail with busy errors.
To resolve this issue, make sure the database is in mount mode before running partial restore from Snap using File System.
Oracle 11.2.0.3 and Higher Errors
When using Oracle 11.2.0.3 version or higher, if the database ID for auxiliary instance is changed using NID script, the table restore from snap or File System backup copy and database cloning restores will fail with the following error message:
NID-00106: LOGIN to target database failed with Oracle error:
ORA-01017: invalid username/password; logon denied
To resolve this issue, install the Oracle update 13366202 on the destination database.
SP File Creation Fails on the Proxy
Backup copy operations fail with the following error message on the proxy computer:
Failed to create spfile on proxy
To resolve this issue, ensure the following:
-
The listener should be configured and in running state
-
Pfile/spfile should not contain any invalid parameters
-
All directories mentioned in the pfile/spfile must be created correctly on the proxy computer.
Completed with One or More Errors
Backup jobs from Oracle iDataAgent will be displayed as "Completed w/ one or more errors" in the Job History in the following cases:
-
When RMAN Script execution for the backup job completes with warnings.
-
When job is killed after backing up some data.
During offline backups, if the database cannot be opened after a backup. The software reports "Completed w/ one or more errors" for restore jobs in the following cases:
-
During a table restore, if the export or import of table fails.
-
RMAN recovery is completed, but an incorrect open mode is selected for restore.
Catalog Errors during IntelliSnap Backup
During IntelliSnap backup job, you may notice the following catalog errors:
File Name: /opt/commvault/MediaAgent/SnapVolumeMounts/SnapMnt_1_2_26099/ oradata/ONLINE/dbconf.cfg RMAN-07517: Reason: The file header is corrupted
File Name: /opt/commvault/MediaAgent/SnapVolumeMounts/SnapMnt_1_2_26099/ oradata/ONLINE/initONLINE.ora RMAN-07517: Reason: The file header is corrupted
File Name: /opt/commvault/MediaAgent/SnapVolumeMounts/SnapMnt_1_2_26099/ oradata/ONLINE/spfileONLINE.ora RMAN-07518: Reason: Foreign database file DBID: 0 Database Name:
File Name: /opt/commvault/MediaAgent/SnapVolumeMounts/SnapMnt_1_2_26099/ oradata/ONLINE/GalaxyControlFile.Conf RMAN-07517: Reason: The file header is corrupted
These error messages can be ignored. As part of the backup job, files such as spfile, pfile and backup controlfiles are copied to the archive log location. Oracle does not recognize these files as archive log files and hence displays the error messages.
Recover until Time Fails because of a Corrupt Archive Redo Log
Issue
The snapping of the archive log volume may not be consistent if there are high transactions and frequent log switches happening.
Symptom
The Recover Until Time fails with the Oracle error ORA-00353: log corruption near block. This can happen when there are frequent online redo log switches.
Resolution
The subclient needs to have the Use Rman for Log Backup option enabled. Right-click the subclient and select Properties. On the Subclient Properties dialog box, navigate to the Logs Backup tab and select the Use Rman for Log Backup check box. Retry the operation.
ORCL0044: The Oracle Wallet May Not Open after an Offline Backup and Restore
Issue:
If the Oracle Wallet is defined in the default location (ORACLE_BASE) and there are no entries to the ORACLE_WALLET location in the sqlnet.ra file, the Oracle Wallet may not open after an offline backup and restore when either of the following is true:
-
The subclient is configured with no lights-out script (Subclient Properties dialog box, Advanced tab, Offline Arguments tab)
-
The Switch Database Mode for Restore option (Options tab) was used for the restore.
Solution
Use one of the following options to fix this issue.
-
Define the Oracle Wallet location in the sqlnet.ra file
-
Configure a post-script that opens the Oracle Wallet (Subclient Properties dialog box, Pre/Post Process tab, PostBackup Process box)
ORCL0045: An IntelliSnap Backup Copy Fails when the ASM_DISKSTRING Parameter Is Not Set
Issue
An IntelliSnap backup copy fails for an Oracle database that runs in an ASM configuration when the ASM_DISKSTRING parameter is not set.
The error is similar to the following
SQL>alter system set asm_diskstring='/opt/commvault/MediaAgent/SnapVolumeMounts/SnapMnt_1_*/*' * ERROR at line 1: ORA-02097: parameter cannot be modified because specified value is invalid ORA-15014: path '/dev/dm-19' is not in the discovery set
Solution
Set the Oracle ASM_DISKSTRING initialization parameter to a value that allows ASM to discover all of the ASM disks, for example, /dev/dm-*.
ORCL0046: Some Channels Unexpectedly Terminate during a Multistream IntelliSnap Oracle 12c Backup Copy
Issue
Some channels unexpectedly terminate during a multistream IntelliSnap Oracle 12c backup copy before the operation completes.
Solution
Check the value of the PGA_AGGREGATE_LIMIT database parameter and increase it. The minimum default value that Oracle recommends is 2 GB. You can get more information in Oracle Doc ID 1520324.1.
Backup using NetApp cluster mode LUNs fails with error: "Snapshot operation not allowed due to clones backed by snapshots"
For more information, see KB article 84806.