Announcing the 0.6.2 Release
This bugfix release includes various fixes to the Kannika Armory platform. It is recommended to upgrade to this version as soon as possible to benefit from the latest bug fixes.
Bug fixes
Mitigated corrupt index: InvariantsNotUpheld
errors
Under certain conditions,
particularly when backing up a large number of topics,
backup tasks would panic with a corrupt index: InvariantsNotUpheld
error.
This error has been mitigated with this release.
Error logging has been improved as well to provide more context in case similar issues occur again.
Unlike the error message suggests, the issue was not caused by a corrupt index. The process attempted to append messages that had lower offsets than the latest messages in the backup (index). Due to a safety mechanism in Armory to protect the integrity of backups, and specifically to protect the order in which messages are appended to the backup, the error would occur.
It was noted that the core engine correctly reset its offsets to the latest offsets in the backup, but that it still received the very first messages for certain topics. After an extensive investigation, it was discovered that there is an existing race condition in the librdkafka library, where the library would not be fully initialized yet after seeking offsets. The issue was reported in 2019 (!), but hasn’t been fixed yet.
With this release, we have implemented a workaround to mitigate the issue by waiting for the library to be fully initialized before starting to consume messages. A better, more permanent fix will be implemented in a future release of Armory.
Keys are now mapped when using Schema Mapping
When restoring data while using Schema Mapping, keys will now also be mapped whenever possible. This was a gap in the previous implementation, as only the payloads were mapped. This change will make it easier to restore data that uses schemas for keys.
Previously,
the operator would only enable the payload-schema-mapping
plugin.
With this release,
the key-schema-mapping
plugin is loaded as well,
thus ensuring that keys are also mapped when restoring data.
Added the all
category to the Kubernetes CRD schemas
When Kannika Armory Custom Resource Definitions (CRDs) are deployed using ArgoCD,
an empty categories: []
field would cause sync issues,
as ArgoCD removes the categories
field when it is empty.
This has been resolved by adding the all
category to each CRD.
This will also ensure that all resources are listed when running kubectl get all
,
and will make it easier to find all resources created by Kannika Armory.
Other Changes
- Fixed an example in the documentation related to schema mapping in ConfigMaps.
Breaking Changes
No breaking changes are present in this release.
Wrapping up
We are pleased with the direction that Kannika Armory is heading in with this release.
We would love to hear your feedback on these new changes. Please try it out and let us know what you think! You can reach out to us on Slack if you have any questions or feedback.
In the next release, we will be focusing on the following features:
- Kafka Event Hub. A new Kubernetes resource to define connections to Kafka and removing the legacy Endpoint resources.
- The Kannika Armory console. We will release a first version of a new web-based console to manage your backups and restores.
- OpenShift support. We will add support for OpenShift as a platform to run Kannika Armory on.
We hope you enjoy this release, and we are looking forward to hearing your feedback!
Changelog 0.6.2
Bug Fixes
- Sidestep librdkafka race condition bug
- Add category
all
to all CRDs - Configure
key-schema-maping
plugin when usingschemaMappingFrom
in a Restore
Documentation
- Fixed schema mapping example
Miscellaneous Tasks
- Improve logging when an invalid index error occurs
- Bump version to 0.6.2