Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-2765

HadoopInputFormatIO should support custom key/value coder

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • 2.2.0
    • None
    • io-java-hadoop-format
    • None

    Description

      Right now HadoopInputFormatIO infers coders with getDefaultCoder but this doesn't work for cases like AvroCoder with GenericRecord, for example, when Avro is used with Parquet.

      https://github.com/apache/beam/blob/master/sdks/java/io/hadoop/input-format/src/main/java/org/apache/beam/sdk/io/hadoop/inputformat/HadoopInputFormatIO.java#L288

      Attachments

        Activity

          People

            Unassigned Unassigned
            sinisa_lyh Neville Li
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: