Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-33197

PyFlink support for ByteArraySchema

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 1.17.0
    • None
    • API / Python
    • None

    Description

      Currently in Python Flink API, when reading messages from a Kafka source, only SimpleStringSchema is available.
      If the data is in arbitary binary format(e.g. marshalled Protocol Buffer msg) it may not be decodable with the default 'utf-8' encoding. 
      There's currently a workaround which is to manually set the encoding to 'ISO-8859-1' which supports all possible byte combinations. 
      However this is not an elegant solution.
      We should support ByteArraySchema which outputs a raw byte array for subsequent unmarshalling.

      Attachments

        Activity

          People

            Unassigned Unassigned
            liu.chong Liu Chong
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: