Senior Developer Advocate (Presenter)
The workflow of Schema Registry includes:
In this module you will learn about the workflow of using schemas. This workflow starts with writing schema files, adding them to a project, and leveraging tools such as Maven and Gradle to generate the model objects that schemas represent as well as register and update them in a schema registry.
Let’s first look at a schema defined using the Protobuf format.
package
declaration creates a name space for proto files to prevent name clashes. It ends up being the package name in the generated Java class unless you specify an option java_package
in the proto file.option java_outer_classname
field describes the name of the generated java filemessage
in the schema as an inner class in the generated fileMore detail of how Protobuf works is in the next module.
Next, let’s examine what the same schema looks like using the Avro format. Notice that it’s defined in JSON format.
More detail about Avro is in the next module as well.
The schema files you write need to go in your project somewhere. In this course the Avro schema files go in the src/main/avro
directory and Protobuf files land in src/main/proto
.
Avro – Gradle and Maven plugins
Protobuf – Gradle and Maven plugins
Once you’ve written your schema files you will want to generate the model/data objects from the schema files. Fortunately, there are plugins available, either Maven or Gradle, that can integrate into your local build.
This course focuses on how to configure and use the Gradle plugin. In the upcoming exercise you will configure a build.gradle
file to use with Avro and Protobuf schemas.
We should note that Schema Registry supports JSON Schema as well, but that’s more of a specification while Avro and Protobuf are full serialization platforms. As a result the tooling is much better for these two versus JSON Schema. JSON Schema will be covered when we go in-depth on the schema types, but the rest of the course sticks to either Avro or Protocol Buffers.
Let’s now do a quick walkthrough of a Gradle configuration file. The plugins section is where you identify the plugins that will be used when running Gradle scripts. This example includes plugins required for working with Schema Registry.
Here’s how the Protobuf plugin is set up. The generatedFilesBaseDir
points to the directory where the generated files land.
The upcoming exercise will use the Avro plugin defaults so the gradle.build
file doesn’t contain configuration settings for it.
Here you have the schemaRegistry configuration block for the Gradle schema registry plugin.
url
field is set to the endpoint address for the Scheme Registrycredentials
block includes authentication details that allow connectivity with Schema Registry in Confluent Cloud. The credentials included in requests sent to Schema Registry need to be in the form API Key:API Secret
. The Gradle schema registry plugin will form these credentials using the values set for <username>
and <password>
. This will result in <username>:<password>
being included in the request.register
block contains all the information the plugin uses to register a schema.To generate the objects from schema files, run the ./gradlew clean build
command. This will generate the files seen in this illustration. Note that the files shown here don’t necessarily reflect what you will see when you go through the exercise that follows.
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.