Apache Kafka On The Go: Kafka Consumers
What is a Kafka consumer?
A consumer is a client that is used to read data from Apache Kafka.
How do Kafka consumers keep track of their progress?
As consumers read events from a Kafka topic, they record their progress using an offset, which is basically a bookmark. Should a consumer go down for any reason, once it recovers, it uses that bookmark to quickly resume from where it left off.
Do events disappear when they are consumed by a Kafka consumer?
No. One of the main advantages of Kafka is that it’s not just a message queue. In Kafka, messages are durably stored in the cluster for a configurable (potentially infinite!) amount of time. And while the events are in Kafka, there can be any number of consumers consuming those same events whenever they'd like.
How do I configure a Kafka consumer client?
At a minimum, consumers need to know where your Kafka cluster is located within your network—the bootstrap servers—and any connection credentials. Beyond this, there are a number of other optional consumer configurations that may be useful to you.
What programming languages can I use to write a Kafka consumer?
Kafka libraries are available in a huge number of programming languages—C++, Java, and Python, to name a few—meaning that you’re able to write a consumer in any of your favorite languages. There’s one for just about every language you can think of.
Check out a consumer client example.