Stream Quality

Overview

Large systems of streaming data involve multiple teams working on different components. To ensure effective communication between these components, we need an API for our streams. The API often takes the form of a schema. However, if we want these teams to be able to use each other's APIs then we need a central authority where they can be managed and explored. In this video, we will explore the challenge of using APIs across multiple teams, and introduce the Confluent Schema Registry which can be used to manage and enforce schemas.

Topics:

• Cross-Team Compatibility
• Stream APIs
• Schemas
• Confluent Schema Registry

Code

Customer Record: Before

{
	"name":"John Smith",
	"phone":"(123)456-7890"
}

Customer Record: After

{
	"name":"John Smith",
	"phone":[
		{
			"type":"mobile",
			"number":"(123)456-7890"
		},
		{
			"type":"work",
			"number":"(123)987-6543"
		}
	]
}

Customer Schema

{
	"name":"Customer",
	"type":"record",
	"fields":[
	  { "name":"name", "type":"string" },
	  {
	     "name":"phone",
	     "type":{
	        "type":"array",
	        "items":{
	           "type":"record",
	           "name":"PhoneNumber",
	           "fields":[
	              { "name":"type", "type":"string" },
	              { "name":"number", "type":"string" }
	           ]
	        }
	     }
	  }
	]
}

Resources

• Stream Quality on Confluent Cloud