Skip to content
Kannika

Schedule backup snapshots

This tutorial shows how to use a Kubernetes CronJob to automate backup storage rotation, creating timestamped snapshots at regular intervals.

Challenge

Section titled “Challenge”

When backing up topics, the storage preserves all historical data including old record values and tombstones. Retention policies on the Backup do not handle compacted topics, as they work on time-based boundaries rather than key-based compaction.

Scheduled snapshots help you:

  • Create fresh snapshots with current data
  • Achieve faster restore times from smaller, bounded snapshots
  • Maintain a clear audit trail with timestamped backup boundaries
  • Simplify retention policy enforcement

Solution

Section titled “Solution”

A Kubernetes CronJob automates storage rotation by creating new timestamped Storage resources at regular intervals. Each snapshot represents the complete state of your data at the time it was created.

Prerequisites

Section titled “Prerequisites”
  • A Kannika Armory instance available, running on a Kubernetes environment.
  • Local installation of the kubectl binary.

Refer to the Setup section to set up the lab environment.

Scenario

Section titled “Scenario”

In this tutorial, you will simulate a financial services company that needs to meet regulatory compliance requirements:

  • Data: Financial transactions that must be backed up daily
  • Requirement: Daily snapshots with verifiable timestamps for auditors
  • Retention: Clear boundaries enabling 7-year retention policies
  • Recovery: Ability to restore data as it existed on any specific date

The goal is to configure automated daily snapshots that create new timestamped Storage resources, enabling auditors to request data from any specific date and supporting point-in-time recovery.

Setup

Section titled “Setup”

Run the setup script:

Terminal window
curl -fsSL https://raw.githubusercontent.com/kannika-io/armory-examples/main/install.sh | bash -s -- schedule-snapshots

Or clone the armory-examples repository:

Terminal window
git clone https://github.com/kannika-io/armory-examples.git
cd armory-examples
./setup schedule-snapshots

This sets up:

Kubernetes cluster: kannika-kind
├── Namespace: kannika-system
│   └── Kannika Armory
└── Namespace: kannika-data
    ├── EventHub: prod-kafka → kafka-source:9092
    ├── Storage: prod-storage
    └── Backup: prod-backup


Kafka: kafka-source:9092 (localhost:9092)
└── Topic: transactions

Step 1: Create the RBAC resources

Section titled “Step 1: Create the RBAC resources”

The CronJob needs permission to read and update Backups, and to create new Storage resources. Create a ServiceAccount with the required permissions.

Apply the RBAC configuration:

Terminal window
kubectl apply -f tutorials/schedule-snapshots/k8s/backup-storage-rotate.rbac.yaml
# https://github.com/kannika-io/armory-examples/blob/main/tutorials/schedule-snapshots/k8s/backup-storage-rotate.rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: backup-storage-rotate-sa
  namespace: kannika-data
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: backup-storage-rotate-role
  namespace: kannika-data
rules:
- apiGroups: ["kannika.io"]
  resources: ["backups"]
  verbs: ["get", "list", "watch", "patch", "update"]
- apiGroups: ["kannika.io"]
  resources: ["storages"]
  verbs: ["get", "list", "watch", "create", "update"]
- apiGroups: ["apps"]
  resources: ["statefulsets"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: backup-storage-rotate-rb
  namespace: kannika-data
subjects:
- kind: ServiceAccount
  name: backup-storage-rotate-sa
  namespace: kannika-data
roleRef:
  kind: Role
  name: backup-storage-rotate-role
  apiGroup: rbac.authorization.k8s.io

Step 2: Deploy the CronJob

Section titled “Step 2: Deploy the CronJob”

The CronJob performs the storage rotation by:

  1. Disabling the Backup to flush any pending segments
  2. Reading the current Storage configuration
  3. Creating a new Storage with a timestamped name
  4. Updating the Backup to point to the new Storage
  5. Re-enabling the Backup

Apply the CronJob:

Terminal window
kubectl apply -f tutorials/schedule-snapshots/k8s/rotate-backup-storage.cronjob.yaml
# https://github.com/kannika-io/armory-examples/blob/main/tutorials/schedule-snapshots/k8s/rotate-backup-storage.cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: rotate-backup-storage
  namespace: kannika-data
spec:
  schedule: "0 0 * * *"  # Run at midnight daily
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: backup-storage-rotate-sa
          restartPolicy: OnFailure
          containers:
          - name: rotate
            image: alpine:3
            command:
            - /bin/sh
            - -c
            - |
              set -e


              # Install kubectl
              wget -qO /usr/local/bin/kubectl \
                "https://dl.k8s.io/release/$(wget -qO- https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
              chmod +x /usr/local/bin/kubectl


              BACKUP_NAME="prod-backup"
              NAMESPACE="kannika-data"
              TIMESTAMP=$(date +%Y%m%d%H%M%S)


              echo "Starting storage rotation at ${TIMESTAMP}"


              # Step 1: Disable the backup to flush segments
              echo "Disabling backup..."
              kubectl patch backup ${BACKUP_NAME} -n ${NAMESPACE} \
                --type merge \
                -p '{"spec":{"enabled":false}}'


              # Wait for backup StatefulSet to scale down
              echo "Waiting for backup to stop..."
              kubectl wait statefulset -l io.kannika/backup=${BACKUP_NAME} -n ${NAMESPACE} \
                --for=jsonpath='{.status.replicas}'=0 \
                --timeout=60s


              # Step 2: Get current storage configuration
              CURRENT_STORAGE=$(kubectl get backup ${BACKUP_NAME} -n ${NAMESPACE} \
                -o jsonpath='{.spec.sink}')
              echo "Current storage: ${CURRENT_STORAGE}"


              # Extract base name (remove any existing timestamp suffix)
              BASE_NAME=$(echo ${CURRENT_STORAGE} | sed 's/-[0-9]\{14\}$//')
              NEW_STORAGE_NAME="${BASE_NAME}-${TIMESTAMP}"


              echo "New storage name: ${NEW_STORAGE_NAME}"


              # Step 3: Create new storage with timestamp
              # Extract the spec from current storage
              VOLUME_CAPACITY=$(kubectl get storage ${CURRENT_STORAGE} -n ${NAMESPACE} \
                -o jsonpath='{.spec.volume.capacity}')


              echo "Creating new storage..."
              cat <<EOF | kubectl apply -f -
              apiVersion: kannika.io/v1alpha
              kind: Storage
              metadata:
                name: ${NEW_STORAGE_NAME}
                namespace: ${NAMESPACE}
              spec:
                volume:
                  capacity: ${VOLUME_CAPACITY}
              EOF


              # Step 4: Update backup to use new storage
              echo "Updating backup to use new storage..."
              kubectl patch backup ${BACKUP_NAME} -n ${NAMESPACE} \
                --type merge \
                -p "{\"spec\":{\"sink\":\"${NEW_STORAGE_NAME}\"}}"


              # Step 5: Re-enable the backup
              echo "Re-enabling backup..."
              kubectl patch backup ${BACKUP_NAME} -n ${NAMESPACE} \
                --type merge \
                -p '{"spec":{"enabled":true}}'


              echo "Storage rotation complete"
              echo "New storage: ${NEW_STORAGE_NAME}"

Verify the CronJob was created:

Terminal window
kubectl get cronjob rotate-backup-storage -n kannika-data
NAME                    SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
rotate-backup-storage   0 0 * * *   False     0        <none>          10s

Step 3: Test the rotation manually

Section titled “Step 3: Test the rotation manually”

Instead of waiting until midnight, trigger the CronJob manually to verify it works:

Terminal window
kubectl create job --from=cronjob/rotate-backup-storage test-rotation -n kannika-data

Watch the job progress:

Terminal window
kubectl logs -f job/test-rotation -n kannika-data

You should see output similar to:

Starting storage rotation at 20260202120000
Disabling backup...
backup.kannika.io/prod-backup patched
Waiting for backup to stop...
statefulset.apps/prod-backup condition met
Current storage: prod-storage
New storage name: prod-storage-20260202120000
Creating new storage...
storage.kannika.io/prod-storage-20260202120000 created
Updating backup to use new storage...
backup.kannika.io/prod-backup patched
Re-enabling backup...
backup.kannika.io/prod-backup patched
Storage rotation complete
New storage: prod-storage-20260202120000

Step 4: Verify the results

Section titled “Step 4: Verify the results”

Check that the new Storage was created:

Terminal window
kubectl get storage -n kannika-data
NAME                          AGE
prod-storage                  10m
prod-storage-20260202120000   30s

Verify the Backup now points to the new Storage:

Terminal window
kubectl get backup prod-backup -n kannika-data -o jsonpath='{.spec.sink}'
prod-storage-20260202120000

Verify the Backup is running again:

Terminal window
kubectl get backup prod-backup -n kannika-data
NAME         STATUS
prod-backup  Streaming

Examine the new Storage configuration:

Terminal window
kubectl get storage prod-storage-20260202120000 -n kannika-data -o yaml
apiVersion: kannika.io/v1alpha
kind: Storage
metadata:
  name: prod-storage-20260202120000
  namespace: kannika-data
spec:
  volume:
    capacity: 1Gi

Each new Storage resource represents a distinct backup snapshot.

Step 5: Simulate an audit request

Section titled “Step 5: Simulate an audit request”

To demonstrate the compliance benefit, imagine an auditor requests data from a specific date.

List all available snapshots:

Terminal window
kubectl get storage -n kannika-data
NAME                          AGE
prod-storage                  10m
prod-storage-20260202120000   1m

Each timestamped Storage represents data as it existed at that point in time. To restore data from a specific date, you can create a Restore pointing to the corresponding Storage.

Cleanup

Section titled “Cleanup”

To clean up the test job:

Terminal window
kubectl delete job test-rotation -n kannika-data

To remove all tutorial resources:

Terminal window
./teardown

Summary

Section titled “Summary”

In this tutorial, you learned how to:

  1. Set up RBAC permissions for automated storage rotation
  2. Deploy a CronJob that creates timestamped backup snapshots
  3. Verify that each snapshot creates a separate Storage resource
  4. Understand how timestamped snapshots enable compliance auditing

This pattern ensures your organization can:

  • Meet retention requirements: Each snapshot has clear boundaries for retention policies
  • Support audits: Provide verifiable point-in-time data on request
  • Enable recovery: Restore data as it existed on any specific date
  • Automate compliance: No manual intervention required for daily snapshots

For production use, consider adjusting the CronJob schedule to match your compliance requirements (hourly, daily, weekly) and implementing alerting for failed rotations.