EMC Symmetrix (DMX/V-MAX) Storage Guide for IntelliSnap

IntelliSnap Guide Objective and Overview

This guide is an introductory overview of the storage array with its built-in technology and feature sets, along with the licenses and configuration requirements to integrate these controls with the Commvault software. This guide also provides detailed descriptions for configuring hardware-based snapshot mechanisms using the Commvault software.

The IntelliSnap technology of the Commvault software is a modernized approach for data protection, allowing hardware storage systems snapshots to merge directly into the backup process. By automatically integrating application intelligence with array snapshot abilities, Commvault is able to reach through the application and file systems into the storage array to accomplish the following:

• discover volume/disk configurations for the snapshot operations

• coordinate snapshot operations with proper application quiesce

• minimize administrative configuration and eliminating scripting requirements

When a scheduled backup job for a defined application runs, the source system quiesces the selected applications and automatically creates a set of persistent snapshots within the production storage system. With a confirmation of the successful snapshot creation by the host, the workload in the protection job shifts to a secondary proxy server to offload backup operations. This shift releases the production host, which returns to full production side operations. This allows the creation of a consistent data image in minutes with Recovery Point Objectives (RPOs) aligning with the frequency of schedule.

The Proxy completes the second half of the protection job by reaching into the SAN, mounting the snapshot and automatically indexing and cataloging the file-level contents of the snapshot(s). Unlike the hardware-based snapshot approach, Commvault blends the speed and efficiency of array snapshots directly into the backup and restore process, offering full system recovery or single file restore. Once the content-aware indexing completes, the snapshot is retained in the array as a persistent recovery copy to provide a quick recovery option to revert or restore the data volumes.

The Proxy serves a secondary role after the snapshot executes. The same snapshots will mount and copy the relevant file contents from the snapshot and apply deduplication to the data during transport to the backup copy destination. The backup copy employs a separate retention than the snapshot allowing aggressive snapshot retention to preserve tier1 space to meet the RTO/RPO needs. As data moves into the backup copy, the original indices are preserved and stored along with the data to ensure access from any location. Data encryption is also another critical feature to apply to data to keep it highly secured from unwanted eyes.

IntelliSnap supports the leading SAN/NAS storage solutions from Dell EMC, NetApp, LSI, HPE, SUN, IBM and Hitachi Vantara.

EMC Symmetrix Storage Overview

EMC provides customers Symmetrix with the Enginuity^TM platform for Enterprise storage needs.

EMC Clariion is a Storage Subsystem dedicated for high-end LUN-based use. It also does not provide CIFS/NFS capabilities unlike many Network Attached Storage systems. EMC Clariion was the chosen array for the NAS offerings from EMC – Celerra platform.

This guide will provide an in-depth storage technology overview, examples of configuration and necessary steps to configure Commvault with EMC Symmetrix with Enginuity^TM systems. Also, this guide concentrates on the Fixed Block Architecture storage structure (known as Open Systems for Windows and UNIX).

For further EMC Symmetrix training material, please visit powerlink.emc.com.

Architecture of Symmetrix with Enginuity™ and Solutions Enabler

The architecture of EMC Symmetrix is based on the following components:

• Directors/Back-Adapters

• Internal Disk Structure/Virtual Memory

• Enginuity^TM

These components provide the array performance and functionality for today's business needs. The following sections provide a brief description for each component.

Directors/Back-Adapters

EMC provides connectivity between the Virtual Memory and disks through the use of Directors. These Hardware System Boards contain the following, which are all internal to the Symmetrix System:

• CPU

• EPROM/EERPOM/Memory

• connections to I/O ports

Depending on the Enginuity code and Configuration File (IMPL) running on the system, each CPU on a Director can act as one of the following:

• Disk Adapter, which controls the functionality to the raw physical disks (Hard Disk Drives and Solid State Drives).

• Host Adapter, which controls the Fibre Channel and iSCSI connectivity.

• Remote Data Facility Adapter, which controls the Hardware Replication between two Symmetrix Systems.

Back-Adapters are Hardware Board that contains the I/O ports (GBICs, Ethernet and ESCON ports). These boards interact with the Director to ensure the functionality is being fully utilized.

An example of a Full Symmetrix is shown below.

Internal Disk Structure/Virtual Memory

The use of Virtual Memory enables the EMC Symmetrix to align disk drives internally in a Track-based architecture consisting of a single Cylinder and a single Head. Tracks are translated into a lower Block-level disk structure where each working block has a size of 512 bytes for the Fixed Block Architecture to reduce the Virtual Memory layer fragmentation. It also allows easier maintenance of the Virtual Memory to ensure that each Track fits equally into each slot of the Virtual Memory. See the image shown on the right.

For example, if a Read request is sent through the Host Adapter and the adapter determines that the block is missing in Virtual Memory, then the following happens:

• A Read Miss is calculated, which will ask for the entire Track (that contains the block in question) to be brought into the Virtual Memory slot by a Disk Adapter.

• Once the Read from Disk has completed and delivered into the Virtual Memory slot, an internal message (Message Bus) is sent directly to the Host Adapter to continue the Read request on the server that originated the request.

Enginuity™

The layer of code driving Symmetrix systems is called Enginuity. This proprietary code is the name of the storage operating environment that controls components in an EMC Symmetrix storage array. This specialized operating environment is optimized for data storage functions such as:

• providing virtual memory layers

• host interconnectivity

• provisioning of raw physical disks into LUNs for customer’s use on their production/development servers

EMC Engineers (or customers using SYMCLI) are required to load Configuration Files (IMPL) to Directors that will control whether Enginuity serves as a Disk Adapter, Host Adapter, or to provide Replication functionalities.

An example of a Symmetrix model, Enginuity code version, Virtual Memory size and Configuration File (IMPL) checksum is shown below.

Solutions Enabler (SYMAPI)

Solutions Enabler is EMC’s customer software product that allows customers to manage their storage arrays. Solutions Enabler contains a specialized library, or set of formatted commands, which resides on the server/host. To reduce the number of inquiries from the host to the storage array, SYMAPI retains the configuration and status information in a database file named symapi_db.bin.

SYMAPI uses a device called “Gatekeeper” (usually 6 cylinders in size) to execute SCSI commands to retrieve the Symmetrix configuration and store it in the SYMAPI DB.

Solutions Enabler can be used with the following access points:

• GUI interface, known as EMC/Ionix Control Center and Symmetrix Management Console.

• Command Line interface, known as SYMCLI, which generates system calls that creates low-level I/O SCSI commands to the Symmetrix units. See the following example:

  C:\> symcli

  Symmetrix Command Line Interface (SYMCLI) Version V7.2.0.11 (Edit Level: 1108)

  built with SYMAPI Version V7.2.0.11 (Edit Level: 1108)

For further information on which SYMCLI or Ionix Control Center version is supported with different Symmetrix Enginuity^TM version, please visit powerlink.emc.com.

TimeFinder (Local Clone/Snap)

TimeFinder is a Symmetrix feature that allows internal operations in a single Symmetrix system to create point-in-time copies of a data volume without disrupting the source volume.

TimeFinder requires a separate licensing component that is usually provided by EMC to all Symmetrix customers.

The following snapshot capabilities are available:

TimeFinder/Mirror

TimeFinder/Mirror is the traditional way to take a full copy of the source volume by mirroring a BCV (Business Continuance Volume) at a track-by-track level. TimeFinder/Mirror requires a full or incremental synchronization to be completed between the BCV and data volumes before a mirror can be split and reused by another server or presented as another LUN.

TimeFinder/Snap

It provides the ability to take a point-in-time snapshot of the source volume using track-level pointers along with Virtual Device volumes (VDEVs). A VDEV only contains pointers to the data at the point-in-time, thus is much smaller in physical size compared to BCV volumes. Data on the source volume will be copied using the Copy-On-First-Write methodology. If a new write request is received on the source volume, a reference of the old data will be copied to the Log Device pool to provide a reference to the point-in-time reference.

TimeFinder/Clone

It creates full volume copies of production data, allowing you to run simultaneous tasks in parallel on Symmetrix systems. TimeFinder/Clone is used to compress the cycle time for the following processes:

• application testing

• software development

• loading or updating a data warehouse

This significantly increases efficiency and productivity while maintaining continuous support for the production needs of the enterprise.

EMC Array Support

For information on the supported storage for the Commvault software, see Hardware Snapshot Engine Compatibility Matrix.

V-MAX (Symmetrix with Enginuity™ 58xx and above)

Virtual Matrix Architecture (V-MAX) is the latest EMC Symmetrix array which provides storage with fabric-based interconnect between node-based engine/packages (Directors and Physical Memory) enabled by the latest version of EnginuityTM 58xx and above.

With the introduction of V-MAX, the hardware Backplane has been removed and all connectivity between Directors/Memory and I/O ports are done through a high speed switch environment.

DMX (Symmetrix with Enginuity™ 56xx/57xx and below

Direct Matrix Architecture (DMX) is a model of EMC Symmetrix which provides the same functionality of the V-MAX model, with the exception that all Directors, Memory and I/O ports are connected directly to a Backplane. DMX utilizes the available interconnects on the Backplane rather than a high speed interconnect switch.

Symmetrix 5.5 and below

Symmetrix 5.5 and below are deprecated models of Symmetrix. Its functionality is similar to DMX, except that instead of using the Fibre Channel arbitrated loop of disk drives on Disk Adapters, Symmetrix 5.5 (8870) and below uses the old SCSI model.

Integration Requirements for IntelliSnap

Array and Solutions Enabler Software Licenses

The Commvault software requires enabling certain array components and licenses on the EMC Symmetrix system to achieve a successful integration with the software. Refer to the Getting Started for EMC VMAX / Symmetrix documentation for more information.

Supported Applications and Operating Environments

Refer to the IntelliSnap Backup - Support document for a complete list of supported applications and platforms.

Commvault Software

IntelliSnap solutions require the appropriate agents as defined by the customer configuration. See the following terminology for reference in this document:

A CommServe, a necessary storage capacity, and MediaAgents must exist to enable a completely functional solution. See the Getting Started for Dell EMC PowerMax/VMAX3/VMAX/Symmetrix for instructions on:

• CommServe, MediaAgent and File System installation

• Storage Device and Storage Policy Configuration

On top of this basic infrastructure, you can configure the environments described below.

Basic File System Environments

The IntelliSnap base configuration requires the following agents on the Production Host:

• Windows/UNIX File System iDataAgent

• MediaAgent

• VSS Software Provider (Windows Only)

For a configuration where snapshots mount off the host to a Proxy server, implement the following agents on the Proxy:

• File System iDataAgent (Must be similar to Production Host Operating System)

• MediaAgent

Refer to Getting Started - Setup Clients to select Windows/UNIX iDataAgent and perform the required deployment and configuration steps.

Application Environments

Protecting application databases and log volumes through an Array snapshot provide fast access for recovery and many flexible options for backups. IntelliSnap integrates application awareness together with the Array and Commvault to deliver all of the benefits of traditional streaming backups with all of the performance and proxy capabilities of a snapshot. This application awareness allows backups with appropriate log management operations based on the snapshot data.

When implementing for Application Environments simply add the appropriate Application iDataAgent to the IntelliSnap base configuration as follows on the Production Host:

• Windows/UNIX File System iDataAgent

• MediaAgent

• VSS Software Provider (Windows Only)

• Application iDataAgent for selected Application (e.g., Exchange, SQL, Oracle)

For a configuration where snapshots mount off the host to a Proxy server, implement the following agents on the Proxy.

• File System iDataAgent (Must be similar to Production Host Operating System)

• MediaAgent

• Application specific iDataAgent (Must be the same to Production Host) to enable Proxy. This is required for RMAN integrated backups.

• Application API (e.g., Exchange Management Pack, Oracle for RMAN integration)

Refer to Getting Started - Setup Clients to select your application iDataAgent and perform the required deployment and configuration steps.

Virtual Environments

IntelliSnap enables fast protection of large or volatile VMware environments without placing load on the production Virtualization Farm. IntelliSnap technology integration with the Virtual Server Agent enables the Array to perform backups in minutes even with large numbers of virtual machines and sizable data stores. A dedicated ESX server for proxy data movement removes any utilization on the ESX farm providing file and folder recovery from the secondary tier of storage.

To enable IntelliSnap for the Virtual Environment install the following on the physical server(s) or virtual hot-add guest(s):

• Windows/UNIX File System iDataAgent

• MediaAgent

• Virtual Server Agent (VSA)

Refer to Getting Started - VMware for deployment and configuration steps.

IntelliSnap Array Configuration Details with Symmetrix

The Commvault software utilizes three different local replication mechanisms provided by EMC: TimeFinder/Snap, TimeFinder/Mirror and TimeFinder/Clone.

Refer to the Getting Started for EMC VMAX / Symmetrix documentation for array configuration steps.

The IntelliSnap Process

The following sections describe the different IntelliSnap operations.

IntelliSnap Backup Operation

IntelliSnap backups consist of the following operations:

1. The backup job (on-demand or scheduled) starts from the CommCell Console.

2. The file system, associated applications, or Virtual Machines properly acquiesce (VSS calls Windows or through application interfaces such as RMAN the database goes in a Hot Backup mode). In VMware configurations, vStorage APIs are called to create software snapshots and enable delta file creation for each of the guests targeted as contents of the snapshot.

3. Array API is called to:

   • Verify the backup job contents (validating the underlying disk structure for file systems, databases, etc).

     TimeFinder/Mirror establishes the BCV to the Production Volume, waits for synchronization and splits the BCV from the Production volume.

     TimeFinder/Snap creates and actives a Snap session between the Production and Virtual Device volume.

     TimeFinder/Clone creates full volume copies of production data.

   • Change the LUN mapping to allow the BCV and VDEV devices to mount to the source/proxy.

   • Mount the snapshot on the source or the selected proxy computer for post-snapshot operations (e.g., Scan and Catalog for File System). For VMware and RMAN proxy configurations, the Virtual Machines and database files are registered by the proxy application software.

4. Unmount the snapshot and BCV/VDEV devices.

This snapshot now provides availability for backup copy operations and high speed restore, mount and revert operations.

Backup Copy Operation

A backup copy operation provides the capability to copy the snapshots to media and can be useful for creating additional standby copies of data. When you enable backup copy, the snapshots are copied to media (based on the rules specified for the snapshot copy) during the IntelliSnap backup or at a later time.

During the Backup Copy operation:

• The snapshots are copied to media in a sequential order.

• The snapshots are mounted to the source or proxy computer. The mounted snapshot receives commands to scan, and backups like a normal file system reading the required contents.

• The file system backup is performed in the Primary Copy of the storage policy of all defined files. The data is indexed and linked back to the original production host.

• When the backup copy job is finished, the snapshot is unmounted and retained based upon Snapshot Copy retention settings.

Proxy Configuration

IntelliSnap provides a modernized architecture for handling backup operations within the datacenter. Proxy capabilities enable an array-based snapshot to mount off the host eliminating backup processes on the production servers. Each operating system with a IntelliSnap client requires a similar operating system for proxy execution. Commvault will automatically link indexing information back to the original host enabling full application protection for recovery purposes. Execution of application integrity checks may also occur on the Proxy servers to validate the data prior to backup creation.

For a configuration where snapshots mount off the host to a proxy server, deploy the following agents on the proxy server:

• File System iDataAgent (Must be similar to Production Host Operating System)

• MediaAgent

• Application specific binaries for proxy (e.g., Exchange Management Pack, Oracle for RMAN, etc)

Verify Configuration Using SnapTest Tool

You can validate the IntelliSnap configuration prior to running production jobs using the SnapTest utility. Refer to the IntelliSnapBackup - SnapTest Tool documentation for usage information.

Security and Storage Policy Best Practices

Security Roles

The Storage Array technology can be potentially dangerous without proper controls. Typical script based tools lack these controls and expose environments to high risk side effects with very little oversight or reporting. A single miss-aligned scripted argument could cause massive data loss.

Rather than risking the business with scripts or standalone tool sets, the embedded role based security system that Commvault provides allows you to entrust critical actions to the right users at the right time. In most medium to large environments, application, backup, and audit responsibilities may be distributed functions that need to be coordinated into a single policy.

For Example, a customer may have three specific roles within an operations environment:

• Backup Administrators. They are the day to day operators with access to perform standard backup and recoveries, manage media, issue reports, etc.

• Application owners (or Database Administrators). They manage the applications running the business such as application level recoveries and array-based operations.

• Audit & Business Compliance leads. They provide security and process proof of who can perform what and how.

Specific roles should be defined for the IntelliSnap and Application iDataAgents within the CommCell. The following table is an example of the Commvault Security Roles basic structure:

Storage Policies

Managing proper retention on snapshot copies becomes another critical requirement. Improper retention may cause the following:

• increase the amount of tier 1 storage that is holding recovery points.

• cause the snapshots to fall short of fully meeting SLA requirements for the business.

Commvault storage policies are broken down into copies for managing retention on the proper tier of storage. In a typical storage policy for IntelliSnap, three copies will be available:

• Primary snapshot copy

• Primary backup copy (movement to media)

• Offsite disk/tape copy

Storage policy configuration varies from environment to environment. For example, SLAs for Sub 24 hours RPO/RTO drastically lower the returns on leveraging snapshot technology on copies beyond 48 hours. Based on this example, you may set the retention in the following way:

• Primary snapshot copy – 2 days & 0 Cycles

• Primary backup copy – 28 days & 1 Cycle

• Offsite Disk/Tape copy – 60 days & 1 Cycle

This configuration allows snapshot retention on a 48 hour rotation providing multiple high-speed recovery points available on the array to meet the SLA requirement. It also requires storage space allocation to maintain two persistent days of change for the associated clients. By setting “cycles” to 0, the removal of old snapshots occurs regardless of success, so proper alerting and monitoring is required. Improperly setting retention and effects of days and cycles can adversely affect the available recovery scenarios for the business applications.

Managing Snapshots

You can also perform other operations with snapshots such as out of place refresh, single file recoveries, mount and browse capabilities, etc. which provide flexibility to execute daily IT operations. Refer to any of the Advanced documents for an specific Agent for the available operations you can perform to manage snapshots. For example, see IntelliSnap - Advanced Snapshots - VMware.

TimeFinder Mirror/Snap and Commvault Data Aging

TimeFinder/Mirror

The EMC TimeFinder/Mirror is an enterprise-level clone solution to take a full copy of the Production volume to a BCV volume. It allows 1-to-Many relationships between the production volume and the BCV volume. See image on the right for an example.

Each job will behave differently during the cataloging phase as required by the Application iDataAgent during backup. Since each BCV volume contains a single job, the very first IntelliSnap job against the production volume will always be a Full. Once the BCV has been attached with the production volume, the BCV must be freed before it can be reused against the production volume. And once the BCV has been freed, Commvault issues an incremental over a full backup against the TimeFinder/Mirror operation when the next IntelliSnap job is initiated.

SYMCLI defaults to a maximum of eight BCV volumes per Production volume. However this can be extended to a maximum count of 16 using the SYMCLI_MAX_BCV_PAIRS environment variable.

TimeFinder/Snap

EMC TimeFinder/Snap is a simple solution to take a quick snapshot of Production volume by storing pointers of data in a Virtual Device. It allows for 1-to-Many relationships between the production volume and the VDEV volume. When a IntelliSnap job runs, each job consumes a BCV volume for each production volume, regardless of Full/Incremental/Differential IntelliSnap job.

A maximum of 128 target VDEVs may be used for snapshot from a single production volume. This means you can have up to 128 IntelliSnap jobs for each LUN assigned to the Production Host.