Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8818

beam.io.parquetio.ReadAllFromParquet from compressed tar.gz files

Details

    • Wish
    • Status: Resolved
    • P3
    • Resolution: Fixed
    • 2.16.0
    • Missing
    • io-py-parquet
    • None

    Description

      Hi

      Is it possible to read from tar.gz compressed parquet files? Is there a technical limitation to allow for this to happen? It seems to be hardcoded here to read only UNCOMPRESSED parquet files: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/parquetio.py#L227

      Attachments

        Activity

          People

            Unassigned Unassigned
            ethansiew Ethan Siew
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: