Requirements for File System and Application Backups with QEMU Guest Agent

Note

While Commvault Cloud software initiates the backup process by triggering a snapshot of the VM, it does not directly create file- or app-consistent OpenStack backups. QGA (Qemu Guest Agent) is entirely responsible for ensuring the consistency of these snapshots. Proper configuration of QGA is required, and this process varies depending on the OS and distribution

Step 1: Set Metadata Properties

Configure the following metadata properties on the Glance image. These properties will ensure the correct behavior for file system and application-consistent snapshots.

Enable the hw_qemu_guest_agent property in the image metadata to allow OpenStack to communicate with the QEMU Guest Agent (QGA) inside the VM. Without this key, the QGA will not function, and file system freeze/thaw operations will be disabled.
Set the hw_require_fsfreeze property to yes to direct OpenStack to instruct the QGA to freeze the file system before taking a snapshot. This ensures the snapshot is file system-consistent by pausing all write operations, maintaining data integrity.

If the file system cannot be frozen (such as the QGA is not functioning), the snapshot will still be created, but it will be crash-consistent as a fallback.
The os_require_quiesce property attempts to quiesce applications by pausing running processes to ensure all in-memory application data is flushed to disk. It is useful for databases or applications with active transactions. To implement full application consistency, integrate application-specific quiescing scripts (FLUSH TABLES for MySQL or CHECKPOINT for PostgreSQL) with the QGA. If this property is set but quiescing fails (if the host cannot communicate with the guest agent or if the agent is unable to quiesce the OS), the snapshot creation will fail.

You can set these properties using the Horizon Dashboard or OpenStack CLI:
```
openstack image set --property custom_property=yes image_id
```
Example:
```
openstack image set --property hw_qemu_guest_agent=yes image_id
```

Step 2: Configuring Instances

Creating New Instances

For new instances, ensure that the necessary metadata properties are set on the image before instance creation so that libvirt is aware of the QGA’s availability.

Verify that the metadata properties (hw_qemu_guest_agent, hw_require_fsfreeze, and os_require_quiesce) are set on the Glance image according to customer requirements.
After confirming the properties are in place, create a new instance using the image.

Configuring Existing Instances

If an instance has already been created without the necessary metadata properties, it can be reconfigured to use file system and application-consistent backups by following these steps:

Add the required metadata properties to the image used to create the instance, ensuring the QGA and quiescing functionalities are enabled.
Rebuild the instance using the updated image. Rebuilding an instance reapplies the updated image, allowing the instance to inherit the new metadata properties.

Warning

Rebuilding an instance replaces its root disk.

For volume-based instances, the image properties need to be enabled in the base image of the volume, as the properties will apply to the instances created from that volume. If not set, rebuild the instance using an image with these properties enabled.

Step 3: Verifying QEMU Guest Agent on the Instance

Once the instance is running, log in to the VM and verify whether the QGA is active and running:

Log into the Guest VM.
Check the status of the QGA service:
```
sudo systemctl status qemu-guest-agent
```
If the service is active (running), the guest agent is functioning properly. If it is not installed or running, follow the steps below to install and enable it:
1. Install the QGA (if not installed):
```
# For Debian/Ubuntu
sudo apt-get install qemu-guest-agent
# For RHEL/CentOS
sudo yum install qemu-guest-agent
```
2. Enable and start the QEMU Guest Agent:
```
sudo systemctl enable qemu-guest-agent
sudo systemctl start qemu-guest-agent
```

After completing these steps, the QEMU Guest Agent should be running inside the VM, ensuring file system freeze/thaw operations can occur during snapshot creation.

Step 4: Verifying Snapshot Consistency

After creating a snapshot, it is important to verify whether the QGA successfully triggered the file system freeze and thaw operations inside the guest VM.

To do this, check the journalctl logs for the relevant entries:

Log in to the VM.
Check the journalctl logs for QEMU Guest Agent actions:
```
sudo journalctl -u qemu-guest-agent 
```
Check for the following entries during snapshot creation. These log entries confirm that the QGA performed the file system freeze and thaw operations:
```
info: guest-fsfreeze called 
```
```
info: executing fsfreeze hook with arg 'freeze'
```
```
info: executing fsfreeze hook with arg 'thaw'
```

Considerations

When configuring file system and application-consistent backups for OpenStack instances using the QGA, it is important to note the following conditions:

Uncertain Snapshot Consistency: If the hw_require_fsfreeze property is set but os_require_quiesce is not, snapshots will be created, but it will be unclear whether they are crash-consistent or file-system consistent.
Snapshot Failure on Quiesce Failure: If the os_require_quiesce property is set and the agent fails to quiesce the OS, the snapshot creation will fail. This ensures that only fully consistent snapshots are taken, preventing potential data corruption from inconsistent snapshots.
OpenStack Limitation: OpenStack does not provide a method to externally verify whether the QGA is active inside the VM before taking a snapshot. It is important to manually verify that the QGA is running within the VM to ensure consistent backups.
QEMU Guest Agent Limitations: The QGA ensures file system freeze and thaw operations during snapshot creation. However, it does not handle application-specific quiescing (such as database flushing for MySQL/Postgres). Depending on the specific applications running inside the VM, additional scripts will be required to ensure complete application consistency.