Up until this point in the course exercises, we have used a topic with the default six partitions. As you’ve learned through the course modules, though, partitions are useful in allowing you to break up your topic into manageable chunks that can be stored across multiple nodes in your cluster. Through this exercise, you’ll learn how to create topics with different partitions using the command line interface and see how varying topic partition counts affect the distribution of your data.
Again, we recommend following these exercises in order so that you have everything you need to complete the exercise. If you haven’t already set up your CLI to connect to your Confluent Cloud cluster, take a look at the earlier exercise to catch up.
confluent kafka topic list
You should see only the
confluent kafka topic describe poems
In particular, make note of the
num.partitions value, which is 6.
confluent kafka topic create --partitions 1 poems_1 confluent kafka topic create --partitions 4 poems_4
confluent kafka topic produce poems_1 --parse-key
When prompted, enter the following strings:
1:”All that is gold does not glitter” 2:"Not all who wander are lost" 3:"The old that is strong does not wither" 4:"Deep roots are not harmed by the frost" 5:"From the ashes a fire shall awaken" 6:"A light from the shadows shall spring" 7:"Renewed shall be blad that was broken" 8:"The crownless again shall be king"
Repeat step 4 for the
From the Confluent Cloud Console, view the newly produced messages in both topics. Note that the
poems_1 topic has all eight messages in its single partition…
poems_4 topic has a slightly different distribution.
poems_4—to observe how the messages are distributed differently across these topics.
This should have given you a good idea of how partitions will affect the distribution of data across your topic. The next time you create a topic, think about how many partitions you’ll need!
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.