Announcing the 0.10.0 Release

This new release introduces schema registry restore capabilities, automatic importing of topics into backups, new CLI tooling, and more!

Installation

For new installations, see the Installation guide.

For upgrading existing installations, see the associated Upgrading to 0.10.x guide.

Restoring your schema registry

Kannika Armory allows you to restore schemas to your schema registries.

Schema Registry Restore Wizards — Configure a schema registry restore in the console

A new SchemaRegistryRestore is available in the Kannika Armory Custom Resource Definitions (CRDs). This builds on top of the existing SchemaRegistryBackup resource, introduced in the 0.9.x release.

apiVersion: kannika.io/v1alpha
kind: SchemaRegistryRestore
metadata:
  name: confluent-registry-restore
spec:
  enabled: true
  registry: confluent-registry
  registryCredentialsFrom:
    credentialsRef:
      name: "http-basic-creds"
  storage: s3-schema-storage
  storageCredentialsFrom:
    credentialsRef:
      name: "s3-creds"
  importMode: true

This will spawn a new Kubernetes Job that will restore the schemas from the specified storage to the specified registry.

In case of a Confluent Schema Registry the restore process will restore schemas via the Confluent Schema Registry REST API. Both ReadWrite and Import modes are supported.

The console has full support for schema registries backups and restores. You can configure backups and restores for your schema registries directly from the console.

Overview — New sections for schema registry backups and restores

Schema Registry Restore Learn how to configure schema registry restores

Schema Registry Backup Learn how to configure schema registry backups

Schema Registry Learn how to configure schema registries

Automatically import topics in backups

A Backup can automatically discover and import topics at runtime, using regex and glob patterns, defined by topic selectors. This feature is useful when you want to back up topics, without having to manually add each topic to the backup.

The Backup resource has been extended with a new topicSelectors field:

apiVersion: kannika.io/v1alpha
kind: Backup
metadata:
  name: topic-selectors-example
spec:
  source: "my-kafka-cluster"
  sink: "my-storage"
  streams:
    - topic: "magic.events"
  topicSelectors:
    matchers:
    - name:
        literal: "spells.proper" # Matches the topic "spells.proper"
    - name:
        glob: "enchantments.*"   # Matches topics starting with "enchantments."
    - name:
        regex: "^curses\\."      # Matches topics that match the regex "^curses\."

Topic Discovery Learn how to configure topic selectors

Topic feedback in the console

We heard your feedback! The console has been improved to provide better feedback when backing up or restoring topics.

In case of errors, the error message will be shown when hovering a warning icon that appears next to the topic.

Backup Topic Feedback — Feedback in a Backup

Restore Topic Feedback — Feedback during a Restore

Tools to manage your backup

The Armory distribution comes with a CLI tool to help manage backups.

The CLI tool allows you to show information about the backup, or to rebuild the index of the backup.

More tooling will be added in the future.

CLI Tooling Learn how to use the new CLI tooling

View platform and license information in the console

The console can show information about the platform and the license in use. This can be useful to check after upgrading Kannika Armory whether all components have been updated.

Improved resource management in the console

New pages for Credentials, Storages, Event Hubs, and Schema Registries have been added to the console, which allow you to view, update and delete these resources.

References to resources have been made clickable in the console, to help you easily navigate to the details of the resource.

The console now has a collapsable sidebar, which allows you to hide the sidebar to get more space for the main content.

Clone a restore

In case you want to run an existing restore again, you can now easily clone it in the console. All settings will be taken over, and you can start the restore again.

Other changes

The backup status shown in the console now also reflects the actual deployment status. If there is an issue with the deployment of the pod, an error status will be shown.
The operator exposes an /info endpoint on port 8080 that returns information about the license, build, version, configuration, etc.
The console will show a warning if you try to leave a page which contains unsaved changes.
The Helm charts now support setting the amount of replicas using a replicaCount field to make it possible to downscale the components while upgrading the platform.

Performance improvements

Greatly increased metrics ingestion performance to accommodate feedback from large resources

Bug fixes

Copy resource requirements for Schema Registry Backup
Removed GraphQL tracing to prevent a memory leak
Fixed metrics being reset when viewing a Backup after the SSE connection was closed

Breaking changes

Kubernetes: The .spec.config.mapping field has been removed from the Restore CRD. Use .spec.config.topics instead.
Kubernetes: The .spec.partitionRolloverTriggers field has been removed from the Backup CRD. Use .spec.segmentRolloverTriggers instead.
Helm: The operator.config.schemaRegistryBackup.image field has moved to operator.config.schemaRegistryImage.

Upcoming features

Some features that we are working on include:

Backup Worker Groups: We are working on adding support to split a Backup into multiple Pods, so you can spread the workload over multiple nodes.
Preflight checks: Checks will be added to the console to ensure that the configuration is correct before starting a Restore.
Integrated schema mapping in the console: We are adding schema mapping to the console, which will work on top of the current Kubernetes-based schema mapping process.

Please check the Roadmap for more information on upcoming features.

Release Notes

For a full list of changes, see the Changelog.