Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11969

Make row-group size configurable in ParquetIO.Sink

Details

    • Improvement
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.29.0
    • io-java-parquet

    Description

      It doesn't seem that ParquetIO.Sink has an option for setting row-group size. Its builder has a withConfiguration but it does not seem to change rowGroupSize in ParquetWriter.Builder and hence the default 128MB is used. It should be fairly easy to add the plumbing for setting this option here.

      Attachments

        Issue Links

          Activity

            People

              aromanenko Alexey Romanenko
              bashir Bashir Sadjad
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m