Hands-on Exercise: Kafka Connect

Hands On: Kafka Connect

In this exercise, you’ll gain hands-on experience with Kafka Connect by creating a source connector in Confluent Cloud that produces data to Apache Kafka®. Then, you’ll consume this data from the command line.

1. From the Confluent Cloud Console, navigate to your cluster then select "Connectors" in the left-hand side bar

2. Click the "Add Connector" button in the top-right corner.

3. In the search field, enter datagen to narrow down the available connectors. Select Sample Data - Datagen Source.

4. Click the Additional configuration button.

5. Click Add a new topic. Name it inventory, and select "Create with defaults".

6. Select the new inventory topic and click Continue.

7. Create a new API key for the connector to communicate with the Kafka cluster. Select "My account", then "Generate API key & download", and click "Continue".

8. The Datagen source connector can auto-generate a number of predefined datasets. Select the "Inventory" template and set the output format to JSON.

9. Click "Continue" twice to first review the cost of the connector, and again to give it a name (for example InventoryGenerator) and confirm your configuration. It may take a few minutes for the connector to be provisioned.

10. Once the connector is running, open a terminal window and consume messages from the `inventory` topic:

    confluent kafka topic consume --from-beginning inventory

In this exercise, you learned how to quickly get started with Kafka Connect by creating a simple, data-generating source connector. This is just the tip of the iceberg—Kafka Connect is a powerful tool with many capabilities. To explore more, be sure to check out the Kafka Connect 101 course.