Kafka Consumers

What is a Kafka consumer?

A consumer is a client that is used to read data from Apache Kafka.

How do Kafka consumers keep track of their progress?

As consumers read events from a Kafka topic, they record their progress using an offset, which is basically a bookmark. Should a consumer go down for any reason, once it recovers, it uses that bookmark to quickly resume from where it left off.

Do events disappear when they are consumed by a Kafka consumer?

No. One of the main advantages of Kafka is that it’s not just a message queue. In Kafka, messages are durably stored in the cluster for a configurable (potentially infinite!) amount of time. And while the events are in Kafka, there can be any number of consumers consuming those same events whenever they'd like.

How do I configure a Kafka consumer client?

At a minimum, consumers need to know where your Kafka cluster is located within your network—the bootstrap servers—and any connection credentials. Beyond this, there are a number of other optional consumer configurations that may be useful to you.

What programming languages can I use to write a Kafka consumer?

Kafka libraries are available in a huge number of programming languages—C++, Java, and Python, to name a few—meaning that you’re able to write a consumer in any of your favorite languages. There’s one for just about every language you can think of.

Check out a consumer client example.