Introduction

Kannika Armory is a Kubernetes-native backup and restore solution designed specifically for streaming data sources. It enables you to protect your critical streaming infrastructure by creating consistent, point-in-time backups of your Kafka topics and schemas.

Whether you’re safeguarding against accidental data deletion, preparing for disaster recovery scenarios, or migrating data between clusters, Kannika Armory provides the tools you need to keep your streaming data safe and recoverable.

Why Kannika Armory?

Streaming platforms like Kafka are at the heart of modern data architectures, powering real-time analytics, event-driven applications, and mission-critical data pipelines. Losing this data—whether due to human error, infrastructure failures, or disasters—can have severe consequences for your business.

Traditional backup solutions weren’t designed with streaming data in mind. They struggle with the unique challenges of backing up distributed, high-throughput systems where data is constantly in motion. Kannika Armory was purpose-built to address these challenges, providing a backup solution that understands the intricacies of streaming platforms.

Key features

Kannika Armory addresses the challenges of streaming data protection by providing:

• Comprehensive backup coverage: Back up your topic data and schemas independently, giving you full control over what to protect and restore.
• Kubernetes-native design: Manage backups using familiar Kubernetes resources and workflows. Define backup policies as Custom Resources and integrate with GitOps pipelines.
• Flexible restore options: Restore entire clusters or selectively recover specific topics and configurations. Restore to the same cluster, a different cluster, or even a different environment entirely.
• Point-in-time recovery: Roll back to any point in time within your retention window. Accidentally deleted a topic? Restore it to the state it was in five minutes ago, yesterday, or last week.
• Multiple storage backends: Store backups in S3-compatible object storage (AWS S3, MinIO, etc.), Azure Blob Storage, or Google Cloud Storage. Keep your backups close to your infrastructure or replicate them across regions for added resilience.
• Encryption at rest: Protect your backup data with encryption. Kannika Armory supports server-side encryption with your cloud provider’s key management service.
• Minimal operational overhead: Automated scheduling, retention policies, and monitoring keep your backups running without constant attention. Set it up once and let it protect your data around the clock.

How it works

Kannika Armory runs as an operator in your Kubernetes cluster. You define your backup policies using Custom Resources, specifying what to back up and where to store the backups. The operator continuously monitors your Kafka clusters and executes backups according to your configuration.

For example, the following Backup resource backs up a topic named my-topic from a Kafka cluster to a storage location:

apiVersion: kannika.io/v1alpha
kind: Backup
metadata:
  name: my-backup
spec:
  source: my-kafka-cluster           # EventHub to back up from
  sourceCredentialsFrom:
    credentialsRef:
      name: my-credentials           # Credentials for authentication
  sink: my-volume-storage            # Storage to save backups to
  streams:
    - topic: my-topic                # Topic to back up

Full example with EventHub, Credentials, and Storage

# EventHub - defines the Kafka cluster connection
apiVersion: kannika.io/v1alpha
kind: EventHub
metadata:
  name: my-kafka-cluster
spec:
  kafka:
    properties:
      bootstrap.servers: broker:9092
---
# Secret - stores the Kafka credentials
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: kafka-credentials
data:
  username: bXktdXNlcm5hbWU=  # base64 encoded
  password: bXktcGFzc3dvcmQ=  # base64 encoded
---
# Credentials - defines authentication to the Kafka cluster
apiVersion: kannika.io/v1alpha
kind: Credentials
metadata:
  name: my-credentials
spec:
  sasl:
    mechanism: PLAIN
    usernameFrom:
      secretKeyRef:
        name: kafka-credentials
        key: username
    passwordFrom:
      secretKeyRef:
        name: kafka-credentials
        key: password
---
# Storage - defines where backups are stored
apiVersion: kannika.io/v1alpha
kind: Storage
metadata:
  name: my-volume-storage
spec:
  volume:
    capacity: 10Gi
---
# Backup - backs up topics from the EventHub to the Storage
apiVersion: kannika.io/v1alpha
kind: Backup
metadata:
  name: my-backup
spec:
  source: my-kafka-cluster
  sourceCredentialsFrom:
    credentialsRef:
      name: my-credentials
  sink: my-volume-storage
  streams:
    - topic: my-topic

When you need to restore data, you create a Restore resource that specifies what to recover and where to restore it:

apiVersion: kannika.io/v1alpha
kind: Restore
metadata:
  name: my-restore
spec:
  source: my-volume-storage          # Storage to restore from
  sink: my-kafka-cluster             # EventHub to restore to
  sinkCredentialsFrom:
    credentialsRef:
      name: my-credentials           # Credentials for authentication
  enabled: true                      # Start the restore immediately
  config:
    topics:
      - source: my-topic             # Topic name in backup
        target: my-topic             # Topic name to restore to

Full example with EventHub, Credentials, and Storage

# Storage - the source of the restore (where backups are stored)
apiVersion: kannika.io/v1alpha
kind: Storage
metadata:
  name: my-volume-storage
spec:
  volume:
    capacity: 10Gi
---
# EventHub - the target Kafka cluster to restore to
apiVersion: kannika.io/v1alpha
kind: EventHub
metadata:
  name: my-kafka-cluster
spec:
  kafka:
    properties:
      bootstrap.servers: broker:9092
---
# Secret - stores the Kafka credentials
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: kafka-credentials
data:
  username: bXktdXNlcm5hbWU=  # base64 encoded
  password: bXktcGFzc3dvcmQ=  # base64 encoded
---
# Credentials - defines authentication to the Kafka cluster
apiVersion: kannika.io/v1alpha
kind: Credentials
metadata:
  name: my-credentials
spec:
  sasl:
    mechanism: PLAIN
    usernameFrom:
      secretKeyRef:
        name: kafka-credentials
        key: username
    passwordFrom:
      secretKeyRef:
        name: kafka-credentials
        key: password
---
# Restore - restores topics from Storage to the EventHub
apiVersion: kannika.io/v1alpha
kind: Restore
metadata:
  name: my-restore
spec:
  source: my-volume-storage
  sink: my-kafka-cluster
  sinkCredentialsFrom:
    credentialsRef:
      name: my-credentials
  enabled: true
  config:
    topics:
      - source: my-topic
        target: my-topic

Common use cases

• Disaster recovery: Maintain off-site backups that can be used to rebuild your Kafka clusters in the event of a regional outage or catastrophic failure.
• Data protection: Guard against accidental deletion, misconfiguration, or application bugs that corrupt your data. Restore to a known good state in minutes.
• Environment cloning: Create copies of production data for staging, testing, or development environments without impacting your production systems.
• Cluster migration: Move data between Kafka clusters—whether upgrading to a new version, switching providers, or consolidating infrastructure.
• Compliance and auditing: Maintain historical snapshots of your data for compliance requirements or audit purposes.

Architecture

Kannika Armory provides multiple ways to manage your backups: a web-based Console UI, REST and GraphQL APIs, or directly via kubectl. All of these interact with the Kannika Armory Operator running in your Kubernetes cluster.

The operator watches Custom Resources and manages Pods and Jobs that perform the actual backup and restore operations against your Kafka clusters, Schema Registries, and cloud storage.

Custom Resources

Kannika Armory uses Kubernetes Custom Resources to define your backup configuration:

Workload Resources create Pods or Jobs:

• Backup: Continuous backup of Kafka topics to storage
• Restore: Restore data from storage back to Kafka
• SchemaRegistryBackup: Backup schemas from a Schema Registry
• SchemaRegistryRestore: Restore schemas to a Schema Registry

Configuration Resources define connections and authentication:

• EventHub: Connection to a Kafka cluster
• Storage: Where backups are stored (S3, GCS, Azure, or persistent volumes)
• Credentials: Authentication for Kafka (SASL, mTLS)
• SchemaRegistry: Connection to a Schema Registry

These Custom Resources can be managed through multiple interfaces:

The operator watches these Custom Resources and creates the appropriate Kubernetes workloads:

Current platform support

Currently, Kannika Armory supports all platforms that provide a Kafka API, including:

• Apache Kafka
• Confluent Kafka
• Redpanda

Many more integrations are planned.

Quick Install

To install Kannika Armory, follow the steps below. For all requirements, see Requirements. For a more detailed installation guide, see Installation.

Step 1: Install the Custom Resource Definitions (CRDs)

Install the Custom Resource Definitions (CRDs) on your Kubernetes cluster using kubectl.

$ kubectl apply -f https://docs.kannika.io/refs/0.14.0/crd/kannika-crd-v1alpha.yml

Step 2: Install your license

Tip

If you wish to try out Kannika Armory for free, you can request a free trial license.

Install your license to your Kubernetes cluster using kubectl. Replace <YOUR_LICENSE_FILE> with the path to your license file.

$ kubectl create secret generic kannika-license \
  --type=kannika.io/license \
  --namespace kannika-system \
  --from-file=license=<YOUR_LICENSE_FILE>

Upgrading an existing license

To upgrade an existing license, you may simply overwrite the existing license using kubectl apply:

$ kubectl create secret generic kannika-license \
  --namespace kannika-system \
  --from-file=license=<YOUR_LICENSE_FILE> \
  --type=kannika.io/license \
  --dry-run=client -o yaml | kubectl apply -f -

The operator will automatically pick up the license key and check its validity.

Step 3: Install the Helm chart

Install the Helm chart using Helm 3.

$ helm install kannika oci://quay.io/kannika/charts/kannika \
  --namespace kannika-system \
  --create-namespace \
  --version 0.14.0

This will install all required components to your Kubernetes cluster.

Step 4: Protect your data

You’re all set up! Head over to Creating your first backup to secure your Kafka data.