Restoring your first topic
In this guide, you will restore a topic from a Backup to a Kafka cluster. This is a continuation from the Creating your first backup guide, where you created a backup of a Kafka topic.
You will configure a Restore using:
- The Storage where the topic data is backed up to.
- The EventHub where the data will be restored to.
- The Credentials to authenticate with the Kafka cluster.
Pre-requisites
To follow this guide, make sure you have the following tools available:
- A Kubernetes cluster with Kannika Armory installed.
- A running Kafka cluster
- The kubectl CLI tool
- Access to the Kubernetes API.
- A Kubernetes cluster with Kannika Armory installed.
- A running Kafka cluster
- A GraphQL client like Postman.
- Access to the GraphQL API.
- A Kubernetes cluster with Kannika Armory installed.
- A running Kafka cluster
- An HTTP client like Postman.
- Access to the REST API.
Step 1: Draft a Restore
First, you will start with defining a Draft Restore. A Draft Restore contains some basic information about the Restore, but does not contain any topics to restore yet. This allows you to use it as a working document and make changes before starting the Restore.
Create the following Restore resource in Kubernetes:
Configure the restore using the configureRestore
GraphQL mutation:
First,
create the restore using the POST /rest/restores
endpoint:
In this example:
-
The Restore is named
my-restore
. -
The Restore will read data from the Storage
my-volume-storage
. -
The Restore will restore data back to the EventHub
my-kafka-cluster
. -
The Restore will use the Credentials
sasl-creds
to authenticate with the Kafka cluster.
Step 2: Configure the topics
Next, you will configure the topics to restore.
Let’s add the topic my-topic
to the Restore:
Add the following configuration to the Restore:
In this example:
-
The Restore will read the data from
my-topic
, as defined by the key used in themapping
field. which was backed up in the previous guide, and stored in the Storagemy-volume-storage
. -
The Restore will restore the data back to the topic
my-topic
in the EventHubmy-kafka-cluster
, defined by thetarget
field.
Configure the topics to restore using the configureRestoreTopics
GraphQL mutation:
In this example:
Mapping features are not fully supported in the GraphQL API yet.
Add the topics to restore using the POST /rest/restores
endpoint:
In this example:
Mapping features are not fully supported in the REST API yet.
Step 3: Start the Restore
Finally, now that all preparations are done, you are ready to start the restore!
Start the Restore by setting the enabled
field to true
:
Start the restore using the startRestore
GraphQL mutation:
Start the restore using the POST /rest/restores/{id}/start
endpoint:
Step 4: Verify that the Restore is running
To verify that the restore is running, run the following command:
You should see output similar to the following:
You should also see the associated Job running:
You should see output similar to the following:
To verify that the restore is running, run the following query:
You should see output similar to this:
To verify that the restore is running, run the following command:
You should see output similar to this:
Once the restore completes,
all data has been restored to the my-topic
topic in Kafka cluster.
Step 5: Clean up the Restore (optional)
To remove the Restore and its associated resources, simply delete it:
To remove the Restore and its associated resources,
simply delete it using the deleteRestore
mutation:
To remove the Restore and its associated resources,
simply delete it using the DELETE /rest/restores/my-restore
endpoint:
Conclusion
You have successfully restored a Kafka topic from a Backup to a Kafka cluster!
Have a look at the Restoring data from a Backup for a more thorough step-by-step guide on restoring data from a backup, or the Restore reference for more information on the available configuration options.
If you require further assistance, don’t hesitate to reach out to us on Slack!