Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-13157

ParquetIO.parseGenericRecords does not support Configuration

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.35.0
    • io-java-parquet
    • None

    Description

      ParquetIO.Parse currently does not support a Hadoop configuration.

      Although it is possible to set the configuration object, it is ignored later on. Parse delegates to ParseFiles, but misses to pass the configuration object here. Therefore, config is lost.

      Workaround:
      Use ParseFiles directly by unrolling Parse.expand implementation.

      Attachments

        Activity

          People

            michel Michael Luckey
            michel Michael Luckey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3.5h
                3.5h