The guide below demonstrates how to quickly get started with Apache Kafka. You'll connect to a broker, create a topic, produce some messages, and consume them. Be sure to also check out the client code examples to learn more.
Confluent Platform includes Apache Kafka.
Download the community edition by running this command.
wget https://packages.confluent.io/archive/7.3/confluent-community-7.3.2.tar.gz
Once downloaded, run this command to unpack the tar file.
tar -xf confluent-community-7.3.2.tar.gz
For the rest of this quickstart we’ll run commands from the root of the Confluent folder, so switch to it using the cd
command.
cd confluent-7.3.2
We’ll launch the broker in KRaft mode, which means that it runs without ZooKeeper. Run this command first to configure the storage.
./bin/kafka-storage format \
--config ./etc/kafka/kraft/server.properties \
--cluster-id $(./bin/kafka-storage random-uuid)
Now you can launch the broker itself by running this command. Don’t close this terminal window as it will shutdown the broker.
./bin/kafka-server-start ./etc/kafka/kraft/server.properties
N.B. Running Kafka in KRaft mode is currently in preview and not yet recommended for production.
Kafka stores messages in topics. It’s good practice to explicitly create them before using them, even if Kafka is configured to automagically create them when referenced.
Open a new terminal window and cd
into the confluent-7.3.2
directory. Run this command to create a new topic into which we’ll write and read some test messages.
./bin/kafka-topics --bootstrap-server localhost:9092 \
--create \
--topic quickstart
You can use the kafka-console-producer
command line tool to write messages to a topic. This is useful for experimentation, but in practice you’ll use the Producer API in your application code, or Kafka Connect for pulling data in from other systems to Kafka.
Run this command. You’ll notice that nothing seems to happen—fear not! It is waiting for your input.
./bin/kafka-console-producer --bootstrap-server localhost:9092 \
--topic quickstart
Type in some lines of text. Each line is a new message.
When you’ve finished, press Ctrl-C to return to your command prompt.
this is my first kafka message
hello world!
this is my third kafka message. I’m on a roll :-D
Now that we’ve written message to the topic, we’ll read those messages back. Run this command to launch the kafka-console-consumer
.
./bin/kafka-console-consumer --bootstrap-server localhost:9092 \
--topic quickstart \
--from-beginning
As before, this is useful for trying things on the command line, but in practice you’ll use the Consumer API in your application code, or Kafka Connect for reading data from Kafka to push to other systems.
You’ll see the messages that you entered in the previous step.
this is my first kafka message hello world! this is my third kafka message. I’m on a roll :-D
Leave the kafka-console-consumer
command from the previous step running. If you’ve already closed it, just rerun it.
Now open a new terminal window and run the kafka-console-producer
again.
./bin/kafka-console-producer --bootstrap-server localhost:9092 \
--topic quickstart
Enter some more messages and note how they are displayed almost instantaneously in the consumer terminal.
Enter Ctrl-C in the producer and consumer terminals to exit each client program.
Once you’ve finished you can shut down the Kafka broker. Press Ctrl-C in the terminal window in which you started the broker.
Step through the basics of the CLI, Kafka topics, and building applications.
Run pre-built ksqlDB recipes that tackle the highest impact use cases for stream processing
Learn how to route events, manipulate streams, aggregate data, and more.
Write your first application using these full code examples in Java, Python, Go, .NET, Node.js, C/C++, REST, Spring Boot, and further languages and CLIs.
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.