Details
-
Bug
-
Status: Triage Needed
-
P1
-
Resolution: Fixed
-
2.16.0, 2.17.0, 2.18.0, 2.19.0
Description
PR/9275 changed ParDo.getSideInputs from List<PCollectionView> to Map<String, PCollectionView> which is backwards incompatible change and was released as part of Beam 2.16.0 erroneously.
Running the Apache Nemo Quickstart fails with:
Exception in thread "main" java.lang.RuntimeException: Translator private static void org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput) have failed to translate org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27Exception in thread "main" java.lang.RuntimeException: Translator private static void org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput) have failed to translate org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27 at org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:113) at org.apache.nemo.compiler.frontend.beam.PipelineVisitor.visitPrimitiveTransform(PipelineVisitor.java:46) at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:665) at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657) at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657) at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657) at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317) at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251) at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:460) at org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:80) at org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:31) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185) at org.apache.beam.examples.WordCount.main(WordCount.java:192)Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:109) ... 14 moreCaused by: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.ParDo$MultiOutput.getSideInputs()Ljava/util/List; at org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(PipelineTranslator.java:236) ... 19 more