Release 0.18.0
This release focuses on improving existing features and on stability. A lot of work also happened behind the scenes, laying the foundation for the features planned for the next releases.
Read the breaking changes before you upgrade.
Installation
Section titled “Installation”For new installations, see the Installation guide.
For upgrading existing installations, see the associated Upgrading to 0.18.x guide.
Breaking changes
Section titled “Breaking changes”This release contains a few changes that may need action before you upgrade.
Restores fail without Topic:DescribeConfigs
Section titled “Restores fail without Topic:DescribeConfigs”A restore that cannot read the configuration of its target topic now fails that topic and names the missing permission. It previously continued with an empty configuration, or failed later with a misleading plugin error. Check the principal your restores connect with.
Additional properties are rejected on S3 storage endpoints
Section titled “Additional properties are rejected on S3 storage endpoints”Remove sourceAdditionalProps and sinkAdditionalProps from backups and restores that use an S3 endpoint.
Kubernetes 1.30 is the minimum version
Section titled “Kubernetes 1.30 is the minimum version”The Helm chart now enforces the minimum version when installing the platform.
Improvements
Section titled “Improvements”Restores fail early when the topic configuration cannot be read
Section titled “Restores fail early when the topic configuration cannot be read”A restore reads the configuration of its target topic to configure the producer and to initialize its plugins,
which requires the Topic:DescribeConfigs permission.
A restore that lacks it used to go ahead with an empty configuration,
or fail in the topic retention plugin with a message that the topic has no retention.ms property.
It now fails that topic before any plugin is initialized, and reports that it could not read the topic configuration, together with the permission to grant. The other topics in the same restore continue. The other failure reasons a restore reports on a topic carry hints as well, for example that a topic which cannot be found may be hidden from a principal that is not allowed to describe it.
In addition, a new preflight check has been added to the Restore page to accommodate this check.
Failed topic discovery is no longer silent
Section titled “Failed topic discovery is no longer silent”A backup that discovers its topics through selectors retried a failed topic listing forever without saying so. An unreachable or misconfigured event hub therefore looked like a healthy backup with no topics.
Listing failures are now logged as errors from the first attempt. The backup retries for five minutes. If no topic is being streamed by then, the backup stops and reports that it could not fetch topic metadata. A backup that is streaming at least one topic is never stopped by this. It keeps running and only picks up new topics again once discovery recovers, which it logs.
A new job-level backup_discovery_state gauge exposes the discovery state on the pod’s metrics endpoint.
Improved dialogs to add topics
Section titled “Improved dialogs to add topics”The Add topics dialog of a restore showed a single size per topic, without saying whether it was the size of the topic or the space it takes up in the backup storage. It now shows both.
The Add topics dialogs of backups and restores also sort by column, and a topic is selected by clicking anywhere on its row rather than only on the checkbox.
Validation of Kafka properties
Section titled “Validation of Kafka properties”The operator validates the Kafka properties of a backup or restore before deploying it.
An unknown or illegal property sets ConfigurationValid: False naming the offending keys,
and the workload is not deployed or updated until it is removed.
A running backup or restore keeps running on its current configuration.
Reduced memory usage of the monitoring sidecar
Section titled “Reduced memory usage of the monitoring sidecar”The backup monitoring sidecar now limits how much data its Kafka consumer fetches and queues, which keeps its memory usage low for backups with many partitions.
Additional properties for S3 are now invalid
Section titled “Additional properties for S3 are now invalid”The sourceAdditionalProps and sinkAdditionalProps are no longer accepted on backups and restores that use an S3 storage endpoint.
A backup or restore that still carries them reports ConfigurationValid: False naming the keys to remove.
S3 Credentials now follow Kubernetes best practices
Section titled “S3 Credentials now follow Kubernetes best practices”The S3 credentials of a backup or restore are now mounted from the Credentials Secret under /var/run/secrets/kannika.io/,
the conventional location for secret material in Kubernetes.
Nothing changes in your manifests, and the affected pods roll once on upgrade.
Long property values wrap in the General tabs
Section titled “Long property values wrap in the General tabs”Long property values no longer stretch the General tabs of an event hub, backup, or restore beyond the page width. They wrap within the tab instead.
Quieter logs when a broker is unreachable
Section titled “Quieter logs when a broker is unreachable”A backup or restore whose broker is unreachable flooded its logs with the same Kafka client error many times per second, which made the logs expensive to store and hard to read. The first occurrence of an error is still logged as before. Repeats are suppressed, and a summary line with the number of occurrences is logged once per minute for as long as the error persists. A backup also logs when records flow again after consumer errors, so the recovery is as visible as the failure.
Segment timeouts are evaluated on every message
Section titled “Segment timeouts are evaluated on every message”The segment rollover timeout of a Backup is now evaluated on every message, so streams that never go idle roll over on time as well. It previously only fired once a stream became idle, so more data could be added to a segment before flushing it, which the documentation described as intended. A stream receiving a steady trickle of data, for example one message every 500 milliseconds, never went idle for a full second, so the timeout never fired and its slowly filling segment stayed open. Data loss could occur in the very rare scenario where such slow stream was never idle for one second, and such a backup was hit by an abnormal interruption. The Segments page has been updated to reflect this change of behaviour.
Improved property editing
Section titled “Improved property editing”Editing the properties of a resource in the console no longer strips line breaks from multi-line values,
for example a certificate chain with multiple PEM certificates pasted into ssl.ca.pem.
The value field grows with its content and scrolls beyond eight lines.
Improved connection test report when SSL is required
Section titled “Improved connection test report when SSL is required”A connection test reports SSL_REQUIRED when a plaintext connection times out against a TLS listener,
instead of a generic time-out.
Custom CA certificates from a ConfigMap
Section titled “Custom CA certificates from a ConfigMap”The API’s custom CA certificates can now be provided through a ConfigMap
with api.config.tls.customCaCertificates.configMapName,
as an alternative to the existing Secret.
Bug fixes
Section titled “Bug fixes”Invalid lag monitor refresh intervals are rejected
Section titled “Invalid lag monitor refresh intervals are rejected”A Backup with a lagMonitor.refreshInterval below 1s is rejected when the backup starts,
with a message naming the minimum.
Minimum Kubernetes Version enforced by Helm.
Section titled “Minimum Kubernetes Version enforced by Helm.”The umbrella chart declares Kubernetes 1.30 as its minimum version, the floor that has been documented and tested since 0.15.
Fixed internal server error for connection tests
Section titled “Fixed internal server error for connection tests”SASL Credentials resource whose sslConf omits enabled no longer causes an internal server error on the connection test.
Ingestion rates no longer dip below zero
Section titled “Ingestion rates no longer dip below zero”Ingestion rates no longer dip below zero after a worker restart resets the byte counters.
All restored offsets are published again when resuming
Section titled “All restored offsets are published again when resuming”After a restore resumes, the last restored offset of a partition that was fully restored before the interruption is reported again. This fixes the last offsets being -1 in the console.
Operator now tolerates invalid resources
Section titled “Operator now tolerates invalid resources”A single resource the operator could not deserialize into its resource type,
used to stall the operator for every resource of that kind,
causing WatchFailed errors to be logged.
No other resource of that type was reconciled until it was fixed or deleted,
and nothing pointed at the culprit.
Such a resource now reports ConfigurationValid: False naming the offending field,
and the other resources keep being reconciled.
Invalid subject names are tolerated
Section titled “Invalid subject names are tolerated”Schema registry backups and restores now allow subject names with characters outside alphanumerics, dots, dashes, underscores, and slashes.
and will no longer abort the backup with the error Invalid subject name.
Tuning guides
Section titled “Tuning guides”Two new guides collect the settings that determine the memory, CPU, and throughput of a backup and a restore. They explain what each Kafka client property does, how the segment settings interact with memory, how to size the pod resources, and where to start when you have many topics.
Release notes
Section titled “Release notes”For a full list of changes, see the Changelog.