Getting started

To get started with Clumio Protect to back up Apache Iceberg tables on the AWS Glue Data Catalog and Amazon S3 Tables, review the following requirements and information.

Considerations

  • Clumio provides Iceberg-aware backup for Apache Iceberg tables registered in the AWS Glue Data Catalog and for tables in Amazon S3 Tables. What determines support is catalog registration, not the underlying S3 bucket type — Iceberg data in a standard S3 bucket is supported as long as it is registered in the Glue Data Catalog.

  • Tables managed exclusively by other catalogs (for example, a standalone Hive metastore or Unity Catalog) are not supported.

  • Apache Iceberg backups are Iceberg-aware and incremental — Clumio preserves table metadata and snapshot relationships so that restores are automated and transactionally consistent. Backups are stored in the SecureVault Standard tier.

  • Cross-region backup and cross-account/cross-region restore are supported. You can restore to any point in time or snapshot — in place, to a new table, or to an existing empty table.

  • Clumio protects Apache Iceberg tables in all AWS regions where Clumio is available. For the current list, see Supported regions.

  • Backup data is encrypted at rest with a Clumio-managed key by default, or with your own key when Bring Your Own Key (BYOK) encryption is configured. For more information, see Bring Your Own Key (BYOK) encryption.

Supported Iceberg table format versions

Clumio supports the following Apache Iceberg table format versions, for both tables in the AWS Glue Data Catalog and tables in Amazon S3 Tables:

  • Apache Iceberg format version 1

  • Apache Iceberg format version 2

  • Apache Iceberg format version 3

Some tables cannot be protected even when their format version is supported. For more information, see Unsupported Apache Iceberg tables.

Format version 3 capabilities

Format version 3 adds table features that format versions 1 and 2 do not have. Clumio protects a format version 3 table with the same air-gapped, snapshot-level backup as any other Iceberg table. There are no new policy settings and no separate configuration. Clumio preserves the following through a complete backup and restore cycle:

  • Deletion vectors: Format version 3 replaces the positional delete files used in format version 2 with compact binary deletion vectors. Clumio captures and restores them intact, so a merge-on-read table comes back with its row-level deletes applied exactly as they were.

  • Row lineage: The row identifiers and update sequence numbers that format version 3 maintains for change-tracking workflows are carried through backup and restore. On restore, Clumio never reissues a row identifier that was already in use, so downstream change-data consumers stay consistent.

  • Variant columns: Semi-structured, JSON-like variant columns are preserved with their data unchanged, on both AWS Glue and Amazon S3 Tables.

  • Format version upgrades: You can upgrade a table from format version 2 to format version 3 at any time without disrupting its protection, and backups taken before the upgrade remain fully restorable. A restore recreates the table at the format version of the snapshot you select. Choose a snapshot from before the upgrade and the table is restored as format version 2. Choose one from after and it is restored as format version 3.

Unsupported Apache Iceberg tables

Clumio does not protect the following Apache Iceberg tables. Such a table appears on the Inventory page with a protection status of Unsupported, the backup action is not offered for it, and both an on-demand backup and a policy assignment are rejected. You can still unassign a policy from the table.

A Glue table whose data is in another region

A table in the AWS Glue Data Catalog is not protectable when the S3 bucket that holds its data is in a different region than the Glue catalog that holds the table. The bucket that is checked is the one behind the table's metadata location, because that is the pointer Clumio reads the table through.

Clumio blocks these tables instead of backing them up because a restore does not write data back across regions. A backup of such a table would not be restorable, so the table is marked unsupported at inventory time.

When you assign a policy to the table or start an on-demand backup of it, the request is rejected with a message that names both regions, for example:

Iceberg table "sales" is not supported for protection: its data bucket is in region "us-east-1" while its catalog is in region "us-west-2".

To protect the table, do one of the following:

  • Move the table data into the region of the Glue catalog.

  • Register the table in a Glue catalog in the region of the bucket.

Tables in Amazon S3 Tables are not affected. The storage of an S3 table is managed by AWS and always lives in the region of its table bucket.

Note

Clumio checks the region of a table again on its next inventory refresh, which runs once a day. After you move the data or change the catalog, it can take up to a day for the table to become protectable in Clumio.

A table that uses Iceberg native table encryption

A table that uses Apache Iceberg native table encryption is not protectable. Backup and restore of such a table fail with an error stating that Iceberg native table encryption is not supported.

This is the encryption that Iceberg itself records in the table metadata as encryption keys. It is not the same as S3 bucket encryption. A table stored in a bucket encrypted with SSE-S3 or SSE-KMS is fully supported.

Requirements

  • You must have permission to create and update a CloudFormation/Terraform template. For more information, see IAM permissions required to deploy the Clumio service.

  • When you connect your AWS account, enable the Apache Iceberg asset types you want to protect (Glue table, S3 tables, or both).

Procedure

Connect your AWS account

Connect your AWS account and the regions that contain your Apache Iceberg tables to start protecting them with Clumio. When configuring the connection, under Supported asset types, enable Glue table and/or S3 tables under AWS Apache Iceberg. For more information, see Account connection.

Create an Apache Iceberg policy

Create a policy that defines when a backup of your Iceberg tables is generated and how long the backup data is retained. For more information, see Apache Iceberg policies.

Apply the policy to your Iceberg tables

Apply the policy to individual tables, or to multiple tables at once from the Inventory page. For Amazon S3 tables, you can also use tag-based AWS Protection Rules to apply policies automatically across accounts and regions; AWS Protection Rules do not apply to Glue tables, because AWS Glue does not support AWS tags. For more information, see Apache Iceberg backups and AWS Protection Rules.

×

Loading...