S3 Storage

---

This page describes the configuration options for the S3 Storage resource. The S3 Storage resource is used to configure an S3 bucket as a storage backend. It supports AWS S3 as well as S3-compatible storages such as MinIO and Ceph.

Synopsis

apiVersion: kannika.io/v1alpha
kind: Storage
metadata:
  name: s3-storage
spec:
  s3:
    bucket: my-bucket
    region: us-east-1                       # Optional, defaults to us-east-1
    prefix: /path/to/directory              # Optional
    description: "This is an S3 bucket"     # Optional
    endpoint: http://minio:9000             # Optional, for S3-compatible storages
    forcePathStyle: true                    # Optional, defaults to false

Credentials: Access Key and Secret Key

apiVersion: kannika.io/v1alpha
kind: Credentials
metadata:
  name: aws-creds
spec:
  aws:
    description: "Access key authentication for AWS" # Optional description
    accessKeyIdFrom:
      secretKeyRef:
        name: aws-creds          # Name of the secret below
        key: accessKeyId         # References the access key id in the secret
    secretAccessKeyFrom:
      secretKeyRef:
        name: aws-creds          # Name of the secret below
        key: secretAccessKey     # References the secret access key in the secret
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: aws-creds
data:
  accessKeyId: dGhla2V5aWQ= # thekeyid
  secretAccessKey: dGhlU2VjcmV0S2V5 # theSecretKey

Configuring the bucket

To configure the bucket, you need to specify the bucket name and the region.

apiVersion: kannika.io/v1alpha
kind: Storage
metadata:
  name: s3-storage
spec:
  s3:
    bucket: my-bucket
    region: us-east-1
    prefix: /path/to/directory          # Optional
    description: "This is an S3 bucket" # Optional

Caution

Changing the bucket, region or prefix will not move existing data.

Using S3-compatible storages

The S3 Storage resource can be used with S3-compatible storages such as MinIO, Ceph, or any other service that implements the S3 API. To connect to a custom S3-compatible storage, configure the endpoint and forcePathStyle fields.

apiVersion: kannika.io/v1alpha
kind: Storage
metadata:
  name: s3-minio
spec:
  s3:
    bucket: my-bucket
    endpoint: http://minio:9000
    forcePathStyle: true

The endpoint field specifies the URL of the S3-compatible storage. For example, if MinIO is running in the same Kubernetes cluster under the service name minio, the endpoint is http://minio:9000. When endpoint is not set, the standard AWS S3 endpoints are used.

The forcePathStyle field controls how bucket addresses are constructed and defaults to false. S3 supports two addressing styles:

• When forcePathStyle is false (default), virtual-hosted-style addressing is used. The bucket name is part of the hostname, e.g. https://my-bucket.s3.amazonaws.com/key.
• When forcePathStyle is true, path-style addressing is used. The bucket name is part of the URL path, e.g. https://s3.amazonaws.com/my-bucket/key.

Most S3-compatible storages such as MinIO do not support virtual-hosted-style addressing and require forcePathStyle to be set to true.

When using S3-compatible storages, the region field is optional and defaults to us-east-1 if not specified. Some S3-compatible storages ignore the region entirely, but it is still sent as part of the request signing process.

Authentication for S3 storage

Please see the section on AWS Authentication for information on how to configure access to the S3 Storage. Authentication for S3-compatible storages is configured in the same way as for AWS S3.

The following permissions are required for backup processes:

• s3:Put*: required to write backup data and leases.
• s3:Get*: required to read backup data and leases.
• s3:List*: required to list contents of the bucket.
• s3:Delete*: required to remove any leases that are created by the backup process.

The following permissions are required for restore processes:

• s3:Get*: required to read backup data and leases.
• s3:List*: required to list the contents of the bucket.

Depending on your setup, you may need to configure the following IAM policy for the S3 bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Principal": {
        "AWS": "arn:aws:iam::<uid>:user/<username>"
      },
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*",
        "s3:Put*",
        "s3:Delete*"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket_name>",
        "arn:aws:s3:::<bucket_name>/*"
      ]
    }
  ]
}

Replace the <uid> and <username> with the appropriate values for your IAM user. The <bucket_name> should be replaced with the name of your S3 bucket.

Configuring a data retention policy

A retention policy to delete old data can be configured in a Backup.

Make sure the bucket has no lifecycle policy susceptible to delete or move files, or it might interfere with Armory’s own bookkeeping.