Skip to content
Kannika

    Overview

    A Storage resource is used to define where the data of a Backup should be stored, or where a Restore should read the data from.

    Usage

    Storage resources can be managed using the kubectl command line tool, and are available by the name storage or storages.

    Terminal window
    $ kubectl get storages
    NAME            STATUS   AGE   LOCATION
    s3-storage      Ready    1s    s3://kannika-aws-s3/backups
    gcs-storage     Ready    1s    gs://kannika-gcs/backups
    azure-storage   Ready    1s    https://kannika-azure.blob.core.windows.net/backups
    volume-storage  Ready    1s

    Supported Storage

    Currently, the following storage systems are supported:

    Storage Class

    It is also possible to configure a custom StorageClass that can be used in combination with Volume Storage. This allows you to offload your data to any storage provider that is supported by Kubernetes.

    Please check the Storage Class section for more information.

    Objects stored in an Armory Storage

    Kafka Records Backups

    Armory stores different kinds of objects in a storage. The directory structure looks like the following:

    • Directoryprefix optional prefix when defining a Storage
      • Directorytopic.A all topics have their own directory
        • index a file with metadata on this topic’s data
        • ULID_1.kan a segment containing kafka records
        • ULID_2.kan another segment containing kafka records
        • …
        • ULID_N.kan
        • __lease a temporary file to prevent multiple backups from writing to the same storage
        • __time a temporary file to query the storage’s time; used when acquiring a lease on a storage.
      • Directorytopic.B another topic’s directory
        • …

    Here is a high-level description of what they are.

    The index file

    This is the most important object in a topic’s backup. It is a yaml file that contains information on the content of the backup such as which records are stored in a particular .kan file, and expiration dates.

    If a retention period has been defined on a backup, an index may contain references to expired (and deleted) .kan files.

    An index may be rebuilt if it is deleted by accident. However, there is a risk some information (such as expiration times) will be lost in the process.

    .kan files

    These are your topic’s data. Each .kan segment present in the storage should be referenced in the index along with some metadata.

    __lease & __time

    Depending on the storage type you chose, you may find these temporary files in a topic’s directory.

    These are temporary objects used when Backups acquire a lease on a storage. The leasing mechanism prevents multiple Backups from writing to the same location.

    Armory is able to detect orphan leases after some time should a backup fail unexpectedly. Therefore, it is not necessary to remove these files manually if you notice a backup complaining about not being able to acquire a lease. The situation will resolve itself after a few minutes.

    Schema Registry Backups

    Confluent

    The directory structure of confluent schema registry backup is as follows:

    • Directoryprefix optional prefix when defining a Storage
      • Directorysubject.A all subjects have their own directory
        • 1 version 1
        • 2 version 2
      • Directorysubject.B another subject’s directory
        • …

    Subject version files are stored as json documents.