Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9745

[beam_PostCommit_Java_PortabilityApi] Various GCP IO tests failing, unable to deserialize Custom DoFns and Custom Coders.

Details

    Description

      Use this form to file an issue for test failure:

      Initial investigation:

      The bug appears to be popping up on BigQuery tests mostly, but also a BigTable and a Datastore test.

      Here's an example stacktrace of the two errors, showing only the error messages themselves. Source: https://scans.gradle.com/s/c3izncsa4u24k/tests/efn4wciuamvqq-ccxt3jvofvqbe

      java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
      ...
      Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
      ...
      Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
      ...
      Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
      
      
      
      java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
      ...
      Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
      ...
      Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
      ...
      Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
      ...
      Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
      ...
      Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
      ...
      Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
      ...
      Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
      ...
      

      Update: Looks like this has been failing as far back as Apr 4 after a long period where the test was consistently timing out since Mar 31. So it's hard to narrow down what commit may have caused this. Plus, the test was failing due to a completely different BigQuery failure before anyway, so it seems like this test will need to be completely fixed from scratch, instead of tracking down a specific breaking change.


      After you've filled out the above details, please assign the issue to an individual. Assignee should treat test failures as high-priority, helping to fix the issue or find a more appropriate owner. See Apache Beam Post-Commit Policies.

      Attachments

        Issue Links

          Activity

            People

              kenn Kenneth Knowles
              danoliveira Daniel Oliveira
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: