Hedvig Attributes for Bucket Creation

Each bucket in Hedvig maps to a virtual disk. Hedvig-specific attributes, such as disk residence, replication policy, data protection policy, etc., can be associated with a bucket during creation. These attributes are passed on to the underlying virtual disk.

The attributes are specified as <attribute-Key>=<value> pairs, with each pair separated by a colon (:).

aws s3api create-bucket --endpoint <Hedvig_endpoint> --bucket <bucketName> --create-bucket-configuration "LocationConstraint=<<attribute-Key>=<value>>"

The following example creates a bucket with:

  • block size of 4k

  • protection policy of mirroring

  • replication factor of 3

  • replication policy of data center aware (with three data centers)

  • data to reside on Flash

    aws s3api create-bucket --endpoint <Hedvig_endpoint>
       --bucket <bucketName> --create-bucket-configuration
       "LocationConstraint=
       x-amz-meta-hedvig-protection-policy=MIRROR:
       x-amz-meta-hedvig-blocksize=4:
       x-amz-meta-hedvig-replication-factor=3:
       x-amz-meta-hedvig-bucket-disk-residence=Flash:
       x-amz-meta-hedvig-replication-policy-info=snc1\,snc2\,snc3:
       x-amz-meta-hedvig-replication-policy=DataCenterAware"

All attribute keys and their values are listed in the following table.

Important

Most of these attributes cannot be changed after bucket creation. Exceptions are noted in the table.

attribute key

valid values

notes

x-amz-meta-hedvig-protection-policy

MIRROR or ERASURE

Hedvig supports the following Reed-Solomon Erasure Code configurations: RS_4_2, RS_8_2, RS_8_3, and RS_8_4, which stands for:

RS_<datafragments>_<parityfragments>

Erasure Coding has very specific requirements. A minimum of datafragments+parityfragments nodes, racks, or data centers for Agnostic, RackAware, or DataCenterAware policies, respectively, are required.

For example, for an RS_4_2 configuration, you need:

  • a minimum of 6 nodes for an Agnostic policy

  • a minimum of 6 racks for a RackAware policy

  • a minimum of 6 data centers for a DataCenterAware policy

x-amz-meta-hedvig-erasure-config

RS_4_2, RS_8_2, RS_8_3, or RS_8_4

applicable if x-amz-meta-hedvig-protection-policy=ERASURE

x-amz-meta-hedvig-consistency-mode

WEAK or STRONG

x-amz-meta-hedvig-replication-factor

from 1 - 6

applicable if x-amz-meta-hedvig-protection-policy=MIRROR

x-amz-meta-hedvig-replication-policy

Agnostic, DataCenterAware, or RackAware

x-amz-meta-hedvig-replication-policy-info

names of data centers separated by commas

applicable if x-amz-meta-hedvig-replication-policy=DataCenterAware

x-amz-meta-hedvig-blocksize

4 or 64 (in KB)

For Erasure Coding, only a block size of 64 (in KB) is supported.

x-amz-meta-hedvig-bucket-compressed

TRUE or FALSE

x-amz-meta-hedvig-bucket-versioning

unversioned or versioningenabled

After a bucket is created, the versioning status of the bucket can be changed, using the put-bucket-versioning command.

x-amz-meta-hedvig-bucket-disk-residence

HDD or Flash

x-amz-meta-hedvig-bucket-tenant

a tenant to which current user belongs

If not specified, then the user’s primary tenant is the default.

x-amz-meta-hedvig-bucket-quota

from 0 to the available capacity in the primary tenant (in GB)

Specifies the quota for this bucket. See Bucket Quota Management.

Loading...