Skip to content
Kannika

    Announcing the 0.6.1 Release

    This bugfix release includes various fixes for issues introduced in the 0.6.0 release. However, this release also includes documentation covering the plugin system which was already available from 0.4.0 onwards, but which was not yet documented.

    Bug fixes

    Successful finished restores shows as failed instead of done

    The restore status was incorrectly shown as failed when it was successful. This was due to a race condition where the JobFailed condition was being set when the restore was successful. The JobFailed condition has been removed and integrated into the RestoreCompleted condition.

    Change format of uints to ints in CRD schema

    The Kubernetes CRD schema has been updated to use int32 or int64 formats instead of uint32 or uint64 respectively for all number fields. This was done to align with the Kubernetes API conventions, as Kubernetes does not support unsigned integers. This was breaking certain clients that were not able to handle unsigned integers (e.g. the Java Fabric8 client). Tests have been added to ensure that the CRD schema only contains valid formats from now on.

    Fix client key password implementation

    The Kafka Endpoint supports the ssl.key.password configuration since 0.6.0. However, the implementation was incorrect and did not work as expected. This has been resolved.

    Fix labels being removed via the API

    When updating a resource via the API, its labels were incorrectly updated. This was due to the resource not being correctly merged with the existing resource. This was causing some issues. For example, a Restore creating multiple Kubernetes Jobs. This has been fixed by ensuring that the labels are kept when updating the resource.

    Extend your Armory with Plugins

    Kannika Armory has been introducing new weapons to its arsenal since the 0.4.0 release using a plugin system, which was not yet documented up until now. Plugins allow you to inject custom functionality into the core of Kannika Armory, enabling you to extend its capabilities in a way that is not possible with configuration alone.

    Currently, the following plugins are available:

    • Payload Schema Mapping. Allows you to map the payload of your data to a different schema ID.
    • Key Schema Mapping. Allows you to map the keys of your data to a different schema ID.
    • Topic Repartitioning (New). Allows you to change the partitioning strategy of your data. You can use this plugin to repartition your data to a different number of partitions, or to change the partitioning scheme altogether.

    Since v0.4, we’ve been working on an experimental plugin system to inject custom filtering and transformation logic into the backup & restore process. Common “plugins” are statically linked into Armory to avoid the performance hit of sending data back-and-forth between the core engine and external code. Once the API has stabilized, we will provide means to implement your own plugins using WASI (WebAssembly System Interface).

    We will also be revisiting the format of the backup and restore configurations in the near future, and change it to be more flexible and integrated with the plugins, so it will be easier to define plugins for certain topics.

    We are very happy with the possibilities that plugins bring to Kannika Armory. We believe that this will greatly improve flexibility of the platform, and make Kannika Armory a more versatile and powerful tool for your data management needs.

    For more information, head over to the brand-new plugin documentation.

    Repartition your topics

    A new plugin that has been added in this release is the Topic Repartitioning plugin. This plugin allows you to change the partitioning strategy of your data. You can use this plugin to repartition your data to a different number of partitions, or to change the partitioning scheme altogether.

    apiVersion: kannika.io/v1alpha
    kind: Restore
    spec:
      source: "source"
      sink: "sink"
      config:
        plugins:
          - name: topic-repartitioning
            spec:
              target-topic:
                scheme: keyHash
                algorithm: murmur2
                partitions: 10
        mapping:
          source-topic:
            target: target-topic
            disablePreflightChecks: true # Disable partition number check for this mapping

    In this example, the topic-repartitioning plugin is used to repartition the source-topic to target-topic. The target-topic will have 10 partitions, and the partitioning scheme will be based on the key hash using the Murmur2 algorithm.

    For more information, have a look at the Topic Repartitioning plugin documentation.

    Other Changes

    • A RestoreConfigValid condition has been added to the Restore resource to indicate whether the restore configuration is valid. This has been added to accommodate the new uint to int format change in the CRD schema.
    • The possibility to disable pre-flight checks has been added. This is necessary when you require to by-pass certain checks (e.g. when using the Topic Repartitioning plugin to skip the partition number check).
    • The JobFailed condition has been removed from the Restore resource.
    • The RestoreCompleted condition gets the JobFailed reason when a restore job fails.
    • Explicitly set the .spec.enabled field in a Restore resource. This is to help new users understand that they need to set this field to true to starttask the restore.

    Breaking Changes

    • The format of uint fields in the CRD schema has been changed to int.
    • The format of parallelism field on the Restore type in the API has been changed to Int instead of Long.

    Wrapping up

    We are pleased with the direction that Kannika Armory is heading in with this release.

    We would love to hear your feedback on these new features. Please try it out and let us know what you think! You can reach out to us on Slack if you have any questions or feedback.

    In the next release, we will be focusing on the following features:

    • Kafka Event Hub. A new Kubernetes resource to define connections to Kafka and removing the legacy Endpoint resources.
    • The Kannika Armory console. We will release a first version of a new web-based console to manage your backups and restores.
    • OpenShift support. We will add support for OpenShift as a platform to run Kannika Armory on.

    We hope you enjoy this release, and we are looking forward to hearing your feedback!

    Changelog 0.6.1

    Features

    • Add topic-repartitioning plugin
    • Allow disabling pre-flight checks
    • Add restore configuration valid condition to Restore
    • Explicitly set the enabled field in a Restore.

    Bug Fixes

    • Fix client key password implementation
    • Show successful finished restores as done instead of failed
    • Change format of uints to ints in CRD schema
    • Keep labels when updating restore

    Documentation

    • Add plugin system documentation

    Miscellaneous Tasks

    • Bump version to 0.6.1