Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3799

Nexmark Query 10 breaks with direct runner

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 2.4.0, 2.5.0
    • 2.4.0
    • runner-direct

    Description

      While running query 10 with the direct runner like this:

      mvn exec:java Dexec.mainClass=org.apache.beam.sdk.nexmark.Main -Pdirect-runner -Dexec.args="-runner=DirectRunner --query=10 --streaming=false --manageResources=false --monitorJobs=true --enforceEncodability=true --enforceImmutability=true" -pl 'sdks/java/nexmark'

      I found that it breaks with the direct runner with  following exception (it works ok with the other runners):

      [WARNING]
      java.lang.reflect.InvocationTargetException
          at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke (Method.java:498)
          at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
          at java.lang.Thread.run (Thread.java:748)
      Caused by: org.apache.beam.sdk.util.IllegalMutationException: PTransform Query10/Query10.UploadEvents/ParMultiDo(Anonymous) mutated value KV

      Unknown macro: {null, 2015-07-15T00}

      after it was output (new value was KV

      Unknown macro: {null, 2015-07-15T00}

      ). Values must not be mutated in any way after being output.
          at org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.commit (ImmutabilityCheckingBundleFactory.java:134)
          at org.apache.beam.runners.direct.EvaluationContext.commitBundles (EvaluationContext.java:212)
          at org.apache.beam.runners.direct.EvaluationContext.handleResult (EvaluationContext.java:152)
          at org.apache.beam.runners.direct.QuiescenceDriver$TimerIterableCompletionCallback.handleResult (QuiescenceDriver.java:258)
          at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle (DirectTransformExecutor.java:190)
          at org.apache.beam.runners.direct.DirectTransformExecutor.run (DirectTransformExecutor.java:127)
          at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
          at java.util.concurrent.FutureTask.run (FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
          at java.lang.Thread.run (Thread.java:748)
      Caused by: org.apache.beam.sdk.util.IllegalMutationException: Value KV

      Unknown macro: {null, 2015-07-15T00}

      mutated illegally, new value was KV

      Unknown macro: {null, 2015-07-15T00}

      . Encoding was rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAAAAAAAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHAAAAFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAAAAAAAAAASAAB4cgAOamF2YS5sYW5nLkVudW0AAAAAAAAAABIAAHhwdAAHT05fVElNRQ, now rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAAAAAAAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHAAAAFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAAAAAAAAAASAAB4cgAOamF2YS5sYW5nLkVudW0AAAAAAAAAABIAAHhwdAAHT05fVElNRQ.
          at org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.illegalMutation (MutationDetectors.java:144)
          at org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.verifyUnmodifiedThrowingCheckedExceptions (MutationDetectors.java:139)
          at org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.verifyUnmodified (MutationDetectors.java:123)
          at org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.commit (ImmutabilityCheckingBundleFactory.java:124)
          at org.apache.beam.runners.direct.EvaluationContext.commitBundles (EvaluationContext.java:212)
          at org.apache.beam.runners.direct.EvaluationContext.handleResult (EvaluationContext.java:152)
          at org.apache.beam.runners.direct.QuiescenceDriver$TimerIterableCompletionCallback.handleResult (QuiescenceDriver.java:258)
          at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle (DirectTransformExecutor.java:190)
          at org.apache.beam.runners.direct.DirectTransformExecutor.run (DirectTransformExecutor.java:127)
          at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
          at java.util.concurrent.FutureTask.run (FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
          at java.lang.Thread.run (Thread.java:748)
      Mar 07, 2018 5:01:45 PM org.apache.beam.sdk.nexmark.queries.Query10$3 processElement
      INFO: Writing shard-00004-of-00025 with record timestamp 2015-07-15T00:00:09.999Z, window timestamp 2015-07-15T00:00:09.999Z, pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}
      Mar 07, 2018 5:01:45 PM org.apache.beam.sdk.nexmark.queries.Query10$3 processElement
      INFO: Writing shard-00021-of-00025 with record timestamp 2015-07-15T00:00:09.999Z, window timestamp 2015-07-15T00:00:09.999Z, pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}
      Mar 07, 2018 5:01:45 PM org.apache.beam.sdk.nexmark.queries.Query10$3 processElement
      INFO: Writing shard-00018-of-00025 with record timestamp 2015-07-15T00:00:09.999Z, window timestamp 2015-07-15T00:00:09.999Z, pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}
      Mar 07, 2018 5:01:45 PM org.apache.beam.sdk.nexmark.queries.Query10$3 processElement
      INFO: Writing shard-00002-of-00025 with record timestamp 2015-07-15T00:00:09.999Z, window timestamp 2015-07-15T00:00:09.999Z, pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}
      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD FAILURE
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 32.560 s
      [INFO] Finished at: 2018-03-07T17:01:45+01:00
      [INFO] Final Memory: 82M/2831M
      [INFO] ------------------------------------------------------------------------
      [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java (default-cli) on project beam-sdks-java-nexmark: An exception occured while executing the Java class. null: InvocationTargetException: PTransform Query10/Query10.UploadEvents/ParMultiDo(Anonymous) mutated value KV

      Unknown macro: {null, 2015-07-15T00}

      after it was output (new value was KV

      Unknown macro: {null, 2015-07-15T00}

      ). Values must not be mutated in any way after being output. Value KV

      Unknown macro: {null, 2015-07-15T00}

      mutated illegally, new value was KV

      Unknown macro: {null, 2015-07-15T00}

      . Encoding was rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAAAAAAAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHAAAAFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAAAAAAAAAASAAB4cgAOamF2YS5sYW5nLkVudW0AAAAAAAAAABIAAHhwdAAHT05fVElNRQ, now rO0ABXNyADZvcmcuYXBhY2hlLmJlYW0uc2RrLm5leG1hcmsucXVlcmllcy5RdWVyeTEwJE91dHB1dEZpbGUWUg9rZM1SvgIABUoABWluZGV4TAAIZmlsZW5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAxtYXhUaW1lc3RhbXB0ABdMb3JnL2pvZGEvdGltZS9JbnN0YW50O0wABXNoYXJkcQB-AAFMAAZ0aW1pbmd0ADpMb3JnL2FwYWNoZS9iZWFtL3Nkay90cmFuc2Zvcm1zL3dpbmRvd2luZy9QYW5lSW5mbyRUaW1pbmc7eHAAAAAAAAAAAHBzcgAVb3JnLmpvZGEudGltZS5JbnN0YW50Lci-0MYOnM0CAAFKAAdpTWlsbGlzeHAAAAFOjwLrD3QAFHNoYXJkLTAwMDAzLW9mLTAwMDI1fnIAOG9yZy5hcGFjaGUuYmVhbS5zZGsudHJhbnNmb3Jtcy53aW5kb3dpbmcuUGFuZUluZm8kVGltaW5nAAAAAAAAAAASAAB4cgAOamF2YS5sYW5nLkVudW0AAAAAAAAAABIAAHhwdAAHT05fVElNRQ.
      [ERROR] -> [Help 1]
      [ERROR]
      [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
      [ERROR] Re-run Maven using the -X switch to enable full debug logging.
      [ERROR]
      [ERROR] For more information about the errors and possible solutions, please read the following articles:
      [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

       

       

      Attachments

        Activity

          People

            tgroh Thomas Groh
            iemejia Ismaël Mejía
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 40m
                40m

                Slack

                  Issue deployment