Course: Apache Kafka® 101

Producers

5 min
Tim BerglundSr. Director, Developer Advocacy (Course Presenter)

Kafka Producers

The API surface of the producer library is fairly lightweight: In Java, there is a class called KafkaProducer that you use to connect to the cluster. You give this class a map of configuration parameters, including the address of some brokers in the cluster, any appropriate security configuration, and other settings that determine the network behavior of the producer. There is another class called ProducerRecord that you use to hold the key-value pair you want to send to the cluster.

To a first-order approximation, this is all the API surface area there is to producing messages. Under the covers, the library is managing connection pools, network buffering, waiting for brokers to acknowledge messages, retransmitting messages when necessary, and a host of other details no application programmer need concern herself with.

Producer Example

Whether you know it or not yet, you're extremely happy someone wrote this library for you.

try (KafkaProducer<String, Payment> producer = new KafkaProducer<>(props)) {

    for (long i = 0; i < 10; i++) {
        final String orderId = "id" + Long.toString(i);
        final Payment payment = new Payment(orderId, 1000.00d);
        final ProducerRecord<String, Payment> record = 
           new ProducerRecord<>("transactions", 
                                        payment.getId().toString(), 
                                        payment);
        producer.send(record);
   }
} catch (final InterruptedException e) {
    e.printStackTrace();
}

One note: Remember the discussion of partitions up above? Partitions are what take a single topic and break it up into many individual logs that can be hosted on different brokers. Well, it is the producer that makes the decision about which partition to send each message—whether to round-robin keyless messages, compute the destination partition by hashing the key, or apply a custom-configured scheme (although this isn’t very commonly used). In a real sense, partitioning lives in the producer.

Use the promo code KAFKA101 to get $101 of free Confluent Cloud usage

Be the first to get updates and new content

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.