Release 0.19.2

This patch release closes segments after an hour by default, improves how segments are uploaded to S3, and speeds up encoding and decoding segments. Most of the S3 changes concern buckets with a lifecycle rule that aborts incomplete multipart uploads.

Installation

For new installations, see the Installation guide.

For upgrading existing installations, see the associated Upgrading to 0.19.x guide.

Segment uploads to S3

Many S3 buckets have a lifecycle rule that aborts incomplete multipart uploads after a number of days. Armory uploads an open segment as a multipart upload, so such a rule could discard a segment before it was closed.

• A segment only starts a multipart upload once it grows past 5 MiB. Smaller segments are uploaded as a single object when they are closed, so the rule no longer affects them.
• When a part fails to upload, the multipart upload is now aborted, instead of staying open and taking up storage in the bucket. This requires the s3:AbortMultipartUpload permission, see Authentication for S3 storage. Without it, the Backup logs a warning and leaves the upload open, as in earlier releases.
• A NoSuchUpload error now says that S3 discarded the upload, and points at the bucket’s lifecycle rule.

A multipart upload still stays incomplete when a Backup stops without rolling over its open segment, for example because its pod is killed. We recommend a rule that aborts incomplete multipart uploads to remove these, as long as it allows more time than the rollover timeout.

Aborting incomplete multipart uploadsConfigure a lifecycle rule that is safe to use with your backups.

Improvements

• Backup: by default, a segment is now closed after one hour, even when it has not reached the default maximum size of 100 MiB. Backups of quiet topics commit their data sooner, and write more, smaller segments. You can change the rollover timeout with .spec.segmentRolloverTriggers.timeoutSeconds.
• Backup and Restore: records are encoded and decoded faster. Encoding a segment is 25 to 35% faster, and decoding a segment 5 to 15% faster. The segment format is unchanged, so existing backups restore as before.

Release notes

For a full list of changes, see the Changelog.