Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9114

BigQueryIO TableRowParser should support Arrow and Avro data formats

Add voteWatch issue
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-java-gcp
    • None

    Description

      We should implement a function like BigQueryIO#readRows that encapsulate row conversion logic [1] for both Arrow and Avro data formats.

      BigQueryTable#buildIOReader from SQL extension should call the function described above.

       

      [1] https://github.com/apache/beam/pull/10369#discussion_r365979515

      Attachments

        Issue Links

          is a child of

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-8933 BigQuery IO should support reading Arrow format over Storage API

          • P3 - Non-urgent bugs, features, and improvements
          • Resolved

          Activity

            People

              Unassigned Unassigned
              kirillkozlov Kirill Kozlov
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Slack

                  Issue deployment