Here are some of the questions that you may have about Apache Kafka and its surrounding ecosystem.
If you’ve got a question that isn’t answered here then please do ask the community.
Apache Avro is a serialization framework that relies on language agnostic schemas written in JSON, which adhere to its specific format. Avro supports clients written in Java, Python, C, and C#.
Since Avro uses schemas and supports many languages, it's a perfect fit for Schema Registry and Apache Kafka. For more information, read the introduction to Avro and documentation that covers using Avro with Schema Registry.
The Confluent Schema Registry provides a centralized serving layer for your schemas and also provides a RESTful interface for storing and retrieving schemas written in either Avro®, JSON Schema, or Protobuf.
Schema Registry lives outside of and separately from your Kafka brokers. When using it, your producers and consumers still talk to Kafka to publish and read data (messages) to and from topics. But concurrently, they also talk to Schema Registry to send and retrieve schemas that describe the data models for the messages.
Working with schemas guarantees the formats of the objects you produce into and consume from an Apache Kafka broker. But making sure that every developer is using the same schema version can be challenging. Schema Registry gives you a centralized serving layer for your schemas that makes it much easier for everyone to stay in sync.
For a more in-depth explanation on the benefits of using Schema Registry, refer to two blogs by Gwen Shapira: Schemas, Contracts, and Compatibility and Yes, Virginia, You Really Do Need a Schema Registry.
The domain objects in your applications form implicit contracts with all of the developers that work on or interact with them. Schemas make these contracts explicit, reducing the chances of introducing errors when making changes, and thus helping with collaboration.
For a more in-depth explanation read How I Learned to Stop Worrying and Love the Schema.
One good way to check if Schema Registry is running is to use the REST API to run a request. For example, you could list all subjects in Schema Registry:
curl -X GET http://<SR HOST>:8081/subjects
If Schema Registry is up and running, you'll receive a valid response similar to this:
Otherwise you'll get an error.
This hands-on course will show you how to build event-driven applications with Spring Boot and Kafka Streams.