Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-44191

SparkR test failed with R 4.3.0

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsDelete
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.5.0
    • None
    • SparkR
    • None

    Description

      run

       

      build/sbt package -Psparkr
      R/install-dev.sh; R/run-tests.sh 

       

       

       

      ══ Failed ══════════════════════════════════════════════════════════════════════
      ── 1. Error ('test_streaming.R:190:3'): PartitionBy ────────────────────────────
      Error: Error in awaitTermination : streaming query error - Wrong basePath /var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpDbPxOv/sparkr-testcf965edf703d.parquet for the root path: file:/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpBFGJln/sparkr-testd75a4d1118fd.parquet/part-00000-bca8678d-70e2-42ef-abaf-5d5b56d46d62-c000.snappy.parquet
      === Streaming Query ===
      Identifier: [id = 5a6a5714-ea05-473d-bdef-d30db92c8ef6, runId = bcfc36cd-bd4c-4b4a-9693-8ecafa70129a]
      Current Committed Offsets: {}
      Current Available Offsets: {FileStreamSource[file:/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpDbPxOv/sparkr-testcf965edf703d.parquet]: {"logOffset":0}}
      
      
      Current State: ACTIVE
      Thread State: RUNNABLE
      
      
      Logical Plan:
      WriteToMicroBatchDataSourceV1 FileSink[/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T//RtmpDbPxOv/sparkr-testcf961f3f32b4.text], 5a6a5714-ea05-473d-bdef-d30db92c8ef6, [path=/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T//RtmpDbPxOv/sparkr-testcf961f3f32b4.text, checkpointLocation=append], Append
      +- StreamingExecutionRelation FileStreamSource[file:/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpDbPxOv/sparkr-testcf965edf703d.parquet], [name#53978, age#53979, count#53980]
      
      
      Backtrace:
          ▆
       1. ├─SparkR::awaitTermination(q, 5 * 1000) at test_streaming.R:190:2
       2. └─SparkR::awaitTermination(q, 5 * 1000)
       3.   └─SparkR:::handledCallJMethod(x@ssq, "awaitTermination", as.integer(timeout))
       4.     └─base::tryCatch(...)
       5.       └─base (local) tryCatchList(expr, classes, parentenv, handlers)
       6.         └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
       7.           └─value[[3L]](cond)
       8.             └─SparkR:::captureJVMException(e, method)
      
      
      ══ DONE ════════════════════════════════════════════════════════════════════════
      Error: Test failures

       

      ENV:

       

      R --version
      R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
      Copyright (C) 2023 The R Foundation for Statistical Computing
      Platform: aarch64-apple-darwin20 (64-bit)
      
      
      R is free software and comes with ABSOLUTELY NO WARRANTY.
      You are welcome to redistribute it under the terms of the
      GNU General Public License versions 2 or 3.
      For more information about these matters see
      https://www.gnu.org/licenses/.
      
      
      java -version
      openjdk version "1.8.0_372"
      OpenJDK Runtime Environment (Zulu 8.70.0.23-CA-macos-aarch64) (build 1.8.0_372-b07)
      OpenJDK 64-Bit Server VM (Zulu 8.70.0.23-CA-macos-aarch64) (build 25.372-b07, mixed mode) 

       

       

      Run on M2 Max/MacOs 13.4.1

       

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            LuciferYang Yang Jie

            Dates

              Created:
              Updated:

              Slack

                Issue deployment