Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12079

Deterministic coding enforcement causes _StreamToBigQuery/CommitInsertIds/GroupByKey to fail

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.30.0
    • io-py-gcp

    Description

      TypeError: Unable to deterministically encode '<TableReference
 datasetId: 'my-dataset-id'
 projectId: 'my-project-id'
 tableId: 'my-table-id'>' of type '<class 'apache_beam.io.gcp.internal.clients.bigquery.bigquery_v2_messages.TableReference'>', please provide a type hint for the input of 'WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/GroupByKey' [while running 'WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/Map(reify_timestamps)-ptransform-171']

       

      Attachments

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. BEAM-11719 Enforce deterministic coding for GroupByKey and Stateful DoFns

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Triage Needed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. BEAM-12077 Deterministic coding enforcement causes _StreamToBigQuery/CommitInsertIds/GroupByKey to fail

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. BEAM-12078 Deterministic coding enforcement causes _StreamToBigQuery/CommitInsertIds/GroupByKey to fail

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. BEAM-11884 Deterministic coding enforcement causes BigQueryBatchFieldLoads/GroupFilesByTableDestinations to fail

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-11408 GCP BigQuery sink (streaming inserts) uses runner determined sharding

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Triage Needed
          links to

          Web Link GitHub Pull Request #14396

          Web Link GitHub Pull Request #14415

          Activity

            People

              Unassigned Unassigned
              lpost Ludovic Post
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 50m
                  2h 50m