Flink Data Sinks

Overview

In Flink, the endpoint of your datastream takes the form of a data sink. These sinks will usually connect to a database or streaming platform. As data flows through the datastream it eventually gets pushed into the sink through the use of one of Flink's connectors. In this video, we'll introduce the concept of a Flink data sink and show how you can create a simple Kafka sink.

Topics:

• Data sinks
• Connectors
• Kafka serializers
• Connecting a sink to a stream

Code

KafkaRecordSerializer

KafkaRecordSerializationSchema<MyClass> serializer = 
	KafkaRecordSerializationSchema.<MyClass>builder()
		.setTopic("topic_name")
		.setValueSerializationSchema(
			new JsonSerializationSchema<>()
		)
		.build();

KafkaSink

KafkaSink<MyClass> sink = KafkaSink.<MyClass>builder()
	.setKafkaProducerConfig(config)
	.setRecordSerializer(serializer)
	.setDeliveryGuarantee(DeliveryGuarantee.EXACTLY_ONCE)
	.build();

Using a Sink

stream
	.sinkTo(sink)
	.name("sink_name");

Resources

• Data Sinks
• Kafka Sink
• DataStream Connectors