Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23539

Add support for Kafka headers in Structured Streaming

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 3.0.0
    • Component/s: Structured Streaming
    • Labels:
      None

      Description

      Kafka headers were added in 0.11. We should expose them through our kafka data source in both batch and streaming queries. 

      This is currently blocked on version of Kafka in Spark from 0.10.1 to 1.0+ SPARK-18057

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dongjin Dongjin Lee
                Reporter:
                tdas Tathagata Das
              • Votes:
                4 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: