Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7583

Dataflow/Spark/Flink/... post commits failing due to Class interface org.apache.beam.sdk.options.PipelineOptions missing a property named 'exportSummaryToBigQuery'

Details

    Description

      Caused by:
      https://github.com/apache/beam/pull/8822

      Several failing runs:
      https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Spark/2934/
      https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/3049/
      https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/3133/
      https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/3034/

      Example trace:

      12:03:46 Exception in thread "main" java.lang.RuntimeException: org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable to parse query SELECT * FROM PCOLLECTION
      12:03:46 	at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
      12:03:46 	at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
      12:03:46 Caused by: org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable to parse query SELECT * FROM PCOLLECTION
      12:03:46 	at org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:146)
      12:03:46 	at org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.parseQuery(BeamSqlEnv.java:101)
      12:03:46 	at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:124)
      12:03:46 	at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:82)
      12:03:46 	at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
      12:03:46 	at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
      12:03:46 	at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
      12:03:46 	at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery0.expand(SqlQuery0.java:68)
      12:03:46 	at org.apache.beam.sdk.nexmark.queries.sql.SqlQuery0.expand(SqlQuery0.java:51)
      12:03:46 	at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
      12:03:46 	at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
      12:03:46 	at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
      12:03:46 	at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:89)
      12:03:46 	at org.apache.beam.sdk.nexmark.queries.NexmarkQuery.expand(NexmarkQuery.java:33)
      12:03:46 	at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
      12:03:46 	at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
      12:03:46 	at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
      12:03:46 	at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1160)
      12:03:46 	at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
      12:03:46 	at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
      12:03:46 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      12:03:46 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      12:03:46 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      12:03:46 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      12:03:46 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      12:03:46 	at java.lang.Thread.run(Thread.java:748)
      12:03:46 Caused by: org.apache.beam.repackaged.sql.org.apache.calcite.tools.ValidationException: java.lang.IllegalArgumentException: Class interface org.apache.beam.sdk.options.PipelineOptions missing a property named 'exportSummaryToBigQuery'.
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:190)
      12:03:46 	at org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:125)
      12:03:46 	... 25 more
      12:03:46 Caused by: java.lang.IllegalArgumentException: Class interface org.apache.beam.sdk.options.PipelineOptions missing a property named 'exportSummaryToBigQuery'.
      12:03:46 	at org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1608)
      12:03:46 	at org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:114)
      12:03:46 	at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:297)
      12:03:46 	at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.create(PipelineOptionsFactory.java:277)
      12:03:46 	at org.apache.beam.sdk.extensions.sql.impl.rel.BeamEnumerableConverter.createPipelineOptions(BeamEnumerableConverter.java:138)
      12:03:46 	at org.apache.beam.sdk.extensions.sql.impl.BeamCalciteTable.getStatistic(BeamCalciteTable.java:78)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.prepare.RelOptTableImpl.getMonotonicity(RelOptTableImpl.java:340)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.TableNamespace.getMonotonicity(TableNamespace.java:79)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:223)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:994)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:954)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3087)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3069)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3339)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:994)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:954)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:929)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:633)
      12:03:46 	at org.apache.beam.repackaged.sql.org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:188)
      

      Attachments

        Issue Links

          Activity

            People

              riazela Alireza Samadianzakaria
              lcwik Luke Cwik
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: