Details
-
Improvement
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
Currently, the default input collection for KafkaIO.Write is PCollection<KV<K,V>>. To support writing of Kafka headers or different output Kafka topics, we need to change type of input collection to PCollection<ProducerRecord<K,V>>. Also, it will make sense to use ProducerRecord<K,V> instead of KV<K,V> internally in KafkaIO to keep all meta information.
In the same time, we need to keep compatibility for old interface based on KV<K,V> but make it deprecated and totally move to ProducerRecord<K,V> later.