Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-2406

NullPointerException when writing an empty table to BigQuery

Details

    • Bug
    • Status: Resolved
    • P3
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • io-java-gcp
    • None

    Description

      Originally reported on Stackoverflow:
      https://stackoverflow.com/questions/44314030/handling-empty-pcollections-with-bigquery-in-apache-beam

      It looks like if there is no data to write, then WritePartitions will return a null destination, as explicitly stated in the comments:

      https://github.com/apache/beam/blob/v2.0.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WritePartition.java#L126

      But, the ConstantTableDestination doesn't turn that into the constant destination as the comment promises, instead it returns that `null` destination:

      https://github.com/apache/beam/blob/53c9bf4cd325035fabde192c63652ef6d591b93c/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinationsHelpers.java#L74

      This leads to a null pointer error here since the `tableDestination` is that null result from calling `getTable`:

      https://github.com/apache/beam/blob/v2.0.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L97

      Attachments

        Issue Links

          Activity

            People

              reuvenlax Reuven Lax
              bchambers Ben Chambers
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: