0.19.0

This page documents the changes in Kannika Armory version 0.19.0.

Changelog 0.19.0

Features

Backup: topic selectors can match on the properties of a topic’s configuration, such as cleanup.policy or retention.ms, with the Equals, NotEquals, Contains, NotContains, In, NotIn, Exists, DoesNotExist, Gt, and Lt operators. Conditions can be combined with allOf, anyOf, and noneOf groups, in both matchers and excludeMatchers. Matching on properties requires the DescribeConfigs permission on the topics. See Matching on topic properties and Combining conditions.
Backup: when topic selectors match on topic properties, the configuration of discovered topics is described again every 10 minutes, so a topic whose configuration starts matching later is picked up without restarting the backup. Topics that are already being backed up keep running with the selector they matched first.
Backup: each backup worker only describes the configuration of the topics assigned to it, instead of every topic on the cluster.
Backup: how often and how long topic configurations are described can be tuned through environment variables in the Backup’s spec.extraEnvVars: KANNIKA_DISCOVERY_DESCRIBE_CONFIG_INTERVAL_SECS sets the time between describes (default 600), and KANNIKA_DISCOVERY_DESCRIBE_CONFIG_TIMEOUT_SECS sets how long a single describe request may take (default 30). Both take a whole number of seconds; a value that cannot be parsed falls back to the default. See Intervals and timeouts.
Storage: an S3 Storage can trust a private CA through the new caCertificateFrom field, which references a PEM-encoded certificate in a Secret or a ConfigMap. The CA is trusted in addition to the system roots, so an S3-compatible storage behind a private CA can be reached over HTTPS instead of plain HTTP. See Trusting a private CA.
Storage: a backup or restore that cannot verify the TLS certificate of its S3 storage reports that the certificate is not trusted, instead of a generic connectivity error. A CA certificate file that cannot be read at startup is reported with its path and the reason.

Console

Topic selectors of a backup can be edited with a visual editor and a YAML editor, including property conditions and allOf, anyOf, and noneOf groups. The YAML editor shows errors inline.
The CA certificate of an S3 storage can be set from the console, as a PEM value, or as a reference to a Secret or a ConfigMap key. It can be set without a custom endpoint.
Topic rows of a backup show a spinner while a topic is starting or stopping.
When a backup has not reported a topic’s status recently, its row is dimmed and shows when the backup last reported.
The topic page shows the status message and hints of the topic under its heading.
Long topic names are truncated instead of overflowing the topic table.
A backup topic that is backing off shows the reason inline, so it is clear why the topic is not being backed up yet.
The topics of a restore show their status message and hints inline.
The maintenance card uses the theme colors, so it renders correctly in both the light and the dark theme.

Kubernetes API (CRD)

Backup: topicSelectors.matchers and topicSelectors.excludeMatchers accept property conditions, and allOf, anyOf, and noneOf groups.
Storage: added s3.caCertificateFrom, referencing a CA certificate in a Secret (secretKeyRef) or a ConfigMap (configMapKeyRef).
Backup, Restore, SchemaRegistryBackup, and SchemaRegistryRestore: added spec.terminationGracePeriodSeconds, which sets how long their pod gets to shut down gracefully before it is forcibly killed. See Termination Grace Period.

API

GraphQL: the topic selectors of a backup are a tree of rules. TopicSelectorsMatcher and TopicSelectorsMatcherName are replaced by TopicSelectorRule, TopicCondition, TopicNameCondition, TopicPropertyCondition, and TopicSelectorGroup. TopicSelectorsMatcherInput and TopicSelectorsMatcherNameInput are replaced by TopicSelectorRuleInput, TopicConditionInput, TopicNameConditionInput, TopicPropertyConditionInput, and TopicSelectorGroupInput. A name condition is now nested under condition, for example {"condition": {"name": {"glob": "flights.*"}}}.
GraphQL: added the TopicSelectorGroupMatch enum with ALL_OF, ANY_OF, and NONE_OF.
GraphQL: added the TopicPropertyOperator enum.
REST: the topic selectors of a backup accept property conditions, and allOf, anyOf, and noneOf groups. Existing name matchers are unchanged.
The topic preview of a backup matches property conditions.
GraphQL: added the caCertificate field to S3 storages, and to the S3 storage input as a CertificateFieldInput.
GraphQL: added the CertificateRef union of SecretKeyRef and ConfigMapKeyRef, and the ConfigMapKeyRef, ConfigMapKeyRefInput, and CertificateFieldInput types.
REST: added the CA certificate to S3 storages.
GraphQL: added the transition and reportedAt fields to BackupTopicMetrics, and the BackupTopicTransition enum with STARTING and STOPPING.
A topic that has not been reported recently no longer keeps showing as running or starting. A running topic becomes unknown, and a backing off topic becomes failed.

Helm Charts

API: added api.config.backup.metrics.cache.maxSize to configure the maximum number of backups the in-memory metrics cache holds (default: 1000).
API: added api.config.restore.metrics.cache.maxSize to configure the maximum number of restores the in-memory metrics cache holds (default: 1000).
API: added api.config.restore.metrics.scrape.persist.enabled and api.config.restore.metrics.scrape.persist.interval to periodically persist scraped restore metrics to the database (default: enabled, every 5m).
API: added api.config.backup.metrics.startingGrace, how long after a backup pod started its topics are reported as starting while the pod has not reported them yet (default: 60s).
API: added api.config.backup.cache.maxSize and api.config.backup.cache.refreshAfterWrite to configure the in-memory backup cache (default: 100 backups, refreshed after 5s).
API: added api.config.eventhubs.topic.cache.refreshAfterWrite, how long listed event hub topics are served before they are refreshed in the background (default: 60m).
Console: use a faster readiness probe so the console becomes ready sooner after a restart.
Operator: added terminationGracePeriodSeconds under operator.config.pod and under the pod of each pod type, which sets the default termination grace period for pods spawned by the operator. See Default Termination Grace Period.

Bug fixes

Topic status: report a topic as initializing while its backup deployment is still starting, keep a disabled status over an initializing one, and expire stale topic statuses from the cache, so the reported status reflects the actual state of the topic.
Restore: point the recreated-topic hint at the snapshot documentation.
Metrics: invalidate the metrics cache when a backup or restore is deleted, so the console no longer serves stale values for a resource that no longer exists.
Metrics: keep pruning metrics workers after an error instead of cancelling the schedule.
Metrics: raise the cache cap from a hard limit of 25 to 1000 and log evictions, so metrics are no longer dropped when many backups or restores run at the same time.
Metrics: clamp subscription intervals server-side.
Metrics: reload cached session backups without blocking the subscription tick.

Improvements

Metrics: restore metrics are condensed into per-topic samples in a bounded per-restore cache, the same way as backup metrics, and periodically persisted to the database, so they no longer show gaps after the API restarts.
Metrics: the pod scrapers share a single HTTP client instead of each opening their own.
Restore, SchemaRegistryBackup, and SchemaRegistryRestore pods get 60 seconds to shut down gracefully by default instead of 30 seconds, the same as Backup pods.
Schema Registry: back up and restore subjects in named schema contexts. A SchemaRegistryBackup captures subjects from every schema context under their fully qualified name, and a SchemaRegistryRestore restores them into the same context on the target registry. See Schema contexts.
Schema Registry: validate the import mode for each named context that a restore touches.

Documentation

Documented Matching on topic properties and Combining conditions for backup topic selectors.
Documented Trusting a private CA for S3 storages.
Documented schema contexts and deleted subjects for schema registry backups and restores.
Documented the schema registry backup storage layout in Storage.
Documented the Termination Grace Period and the Default Termination Grace Period.
Added Upgrading to 0.19.x guide.

Miscellaneous Tasks

Operator: fail the image build on a missing or invalid license key instead of embedding an empty key that only surfaces as a startup panic.
Bump version to 0.19.0