Loading...

Amazon S3

For Access & Secret Access Keys

  • Authentication

    Access & Secret Access Key - This is the default authentication.

  • Service Host

    A valid endpoint name for the Amazon S3 region provided by the agency.

    Default: s3.[region].amazonaws.com. For example, s3.us-west-1.amazonaws.com.

    To find the region, see https://docs.aws.amazon.com/general/latest/gr/rande.html.

    Note: For Amazon S3 Transfer Acceleration, service host provider name must be provided as s3-accelerate.amazonaws.com.

  • Access Key ID
  • Secret Access Key
  • Bucket

    The following actions must be enabled for the bucket before configuring the library: (sample json file with these actions)

    "s3:CreateBucket",
    "s3:GetBucketLocation",
    "s3:GetObject",
    "s3:PutObject",
    "s3:ListBucket",
    "s3:ListAllMyBuckets"
    "s3:PutObjectTagging",
    "s3:DeleteObject"

    Notes:

    • The CreateBucket permission is required only when the bucket must be created by the MediaAgent while configuring the cloud storage. (This permission can be skipped if an existing bucket is used for configuring the cloud storage.)
    • The ListAllMyBuckets permissions request is required for the Detect button to work.
    • To recall data from Amazon Glacier to S3, make sure that the user associated with the bucket has the RestoreObject permission. For more information on POST Object restore, see https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOSTrestore.html.
  • Storage Class

    The following storage classes are supported:

    • Standard
    • Standard - Infrequent Access
    • One Zone - Infrequent Access
    • Intelligent - Tiering
    • Standard/Glacier (Combined Storage Tiers)
    • Standard-IA/Glacier (Combined Storage Tiers)
    • One Zone-IA/Glacier (Combined Storage Tiers)
    • Intelligent-Tiering/Glacier (Combined Storage Tiers)
    • Standard/Deep Archive (Combined Storage Tiers)
    • Standard-IA/Deep Archive (Combined Storage Tiers)
    • One Zone-IA/Deep Archive (Combined Storage Tiers)
    • Intelligent-Tiering/Deep Archive (Combined Storage Tiers)
    • Glacier
    • Deep Archive
    • Reduced Redundancy Storage

    Reference https://aws.amazon.com/s3/storage-classes/ for more information.

For AWS IAM Role Policy

For AWS IAM Role Policy the selected MediaAgent must reside in the EC2 instance and an IAM Role must be associated with the EC2 instance.

The IAM Role must have the following permissions:

  • Amazon EC2 Full Access
  • Amazon S3 with the following actions: (sample json file with these actions)

    "s3:CreateBucket",
    "s3:GetBucketLocation",
    "s3:GetObject",
    "s3:PutObject",
    "s3:ListBucket",
    "s3:ListAllMyBuckets"
    "s3:PutObjectTagging",
    "s3:DeleteObject"

    Notes:

    • The CreateBucket permission is required only when the bucket must be created by the MediaAgent while configuring the cloud storage. (This permission can be skipped if an existing bucket is used for configuring the cloud storage.)
    • The ListAllMyBuckets permissions request is required for the Detect button to work.
    • To recall data from Amazon Glacier to S3, make sure that the user associated with the bucket has the RestoreObject permission. For more information on POST Object restore, see https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOSTrestore.html.
  • Authentication

    AWS IAM Role Policy - Use this Authentication for an user with the IAM role, thereby allowing the specific user to provide the IAM roles assigned to the user. For more information on IAM Role Policies, refer to https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html.

    Note: For AWS IAM Role Policy the selected MediaAgent must reside in the EC2 instance and an IAM Role must be associated with the EC2 instance. Make sure to select the specific MediaAgent from the drop-down list during library configuration.

  • Service Host

    A valid endpoint name for the Amazon S3 region provided by the agency.

    Default: s3.[region].amazonaws.com. For example, s3.us-west-1.amazonaws.com.

    To find the region, see https://docs.aws.amazon.com/general/latest/gr/rande.html.

  • IAM Role

    Format: <name_of_the_IAMRole>

  • Bucket

    To recall data from Amazon Glacier to S3, make sure that the user associated with the bucket has the RestoreObject permission. For more information on POST Object restore, see https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOSTrestore.html.

  • Storage Class

    The following storage classes are supported:

    • Standard
    • Standard - Infrequent Access
    • One Zone - Infrequent Access
    • Intelligent - Tiering
    • Standard/Glacier (Combined Storage Tiers)
    • Standard-IA/Glacier (Combined Storage Tiers)
    • One Zone-IA/Glacier (Combined Storage Tiers)
    • Intelligent-Tiering/Glacier (Combined Storage Tiers)
    • Standard/Deep Archive (Combined Storage Tiers)
    • Standard-IA/Deep Archive (Combined Storage Tiers)
    • One Zone-IA/Deep Archive (Combined Storage Tiers)
    • Intelligent-Tiering/Deep Archive (Combined Storage Tiers)
    • Glacier
    • Deep Archive
    • Reduced Redundancy Storage

    Reference https://aws.amazon.com/s3/storage-classes/ for more information.

For AWS STS Assume Role

For C2S Access Portal

  • Authentication

    C2S Access Portal - Use this Authentication for a user with credentials to either the Amazon C2S (Amazon Commercial Cloud Services) or Amazon SC2S (Amazon Secure - Commercial Cloud Services).

  • Service Host

    A valid endpoint name for the Amazon S3 region provided by the agency.

    Default: s3.[region].amazonaws.com. For example, s3.us-west-1.amazonaws.com.

    To find the region, see https://docs.aws.amazon.com/general/latest/gr/rande.html.

  • CAP URL

    The CAP URL. For example:

    https://<URL:Port_Name>/TAP/api/v1/credentials?agency=<agency>&mission=<mission>&role=<role>

  • Certificate Filename

    File name provided by the agency. For example: <file_name>.p12.

    Make sure that the file is copied and available in all the MediaAgents using the library under the following folder:

    <software install folder>/Base/Certificates

  • Passphrase

    The password for the certificate file provided by the agency.

  • Bucket

    To recall data from Amazon Glacier to S3, make sure that the user associated with the bucket has the RestoreObject permission. For more information on POST Object restore, see https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOSTrestore.html.

  • Storage Class

    The following storage classes are supported:

    • Standard
    • Standard - Infrequent Access
    • One Zone - Infrequent Access
    • Intelligent - Tiering
    • Standard/Glacier (Combined Storage Tiers)
    • Standard-IA/Glacier (Combined Storage Tiers)
    • One Zone-IA/Glacier (Combined Storage Tiers)
    • Intelligent-Tiering/Glacier (Combined Storage Tiers)
    • Standard/Deep Archive (Combined Storage Tiers)
    • Standard-IA/Deep Archive (Combined Storage Tiers)
    • One Zone-IA/Deep Archive (Combined Storage Tiers)
    • Intelligent-Tiering/Deep Archive (Combined Storage Tiers)
    • Glacier
    • Deep Archive
    • Reduced Redundancy Storage

    Reference https://aws.amazon.com/s3/storage-classes/ for more information.

Best Practices

Amazon S3 - Performance

The performance of S3 reads is better when multiple mount paths are created instead of one single mount path.

Therefore, depending on the estimated Front-End Terabyte (FET) capacity in your environment, create a mount path for every 25 terabyte (TB) of data.

For example if the estimation is 100 TB, then create 4 mount paths.

Note that the same bucket can be used to create the second and subsequent mount paths. A unique base folder will be created for each mount path under the bucket.

In addition, make sure that the Spill and Fill Mount Paths option is enabled in the Library Properties. This setting will help to distribute the objects across different partitions in the bucket, making the retrieval of the objects faster. For more information on this option, see Parameters for Mount Path Usage.

See Also: Cloud Connection Performance Tuning

Related Topics

Last modified: 4/18/2019 3:37:37 PM