Here are some of the questions that you may have about Apache Kafka and its surrounding ecosystem.
If you’ve got a question that isn’t answered here then please do ask the community.
Apache Kafka® is an open source, event streaming platform. It provides the ability to durably write and store streams of events and process them in real time or retrospectively. Kafka is a distributed system of servers and clients that provide reliable and scalable performance.
Learn more about what Kafka is in this free Kafka 101 training course.
Kafka's performance at scale can be attributed to the following design characteristics:
Kafka naturally provides an architecture in which components are decoupled using asynchronous messaging. This design reduces the amount of point-to-point connectivity between applications, which can be important in avoiding the infamous "big ball of mud" architecture that many companies end up with.
Kafka can scale to handle large volumes of data, and its broad ecosystem supports integration with many existing technologies. This makes Kafka a good foundation for analytical systems that need to provide low latency and accurate information.
Event streaming has been applied to a wide variety of use cases, enabling software components to reliably work together in real time.
Kafka has become a popular event streaming platform for several reasons:
Kafka is an event streaming system that may be a good fit anywhere you use a message bus, queuing system, or database. It excels at the real-time processing of data, so it may be an especially good match if all your data matters, but the latest data is particularly important. For example, instant messaging, order processing, warehouse notifications and transportation all need to store and process large amounts of data, and handling the latest data swiftly is essential. (See more example use cases.)
The Kafka broker is written in Java and Scala.
Client libraries are available in many languages and frameworks including Java, Python, .NET, Go, Node.js, C/C++, and Spring Boot. There is also a REST API for interacting with Kafka. Many other languages are available with Kafka community-built libraries.
All technologies have tradeoffs, but in recent years Kafka has seen tremendous adoption as people move away from traditional messaging queues such as IBM MQ, RabbitMQ, and ActiveMQ. Common reasons why people move to Kafka are for its durability, speed, scalability, large ecosystem, and event-stream processing integrations.
Kafka has five core APIs for JVM-based languages:
This hands-on course will show you how to build event-driven applications with Spring Boot and Kafka Streams.