Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35573

Make SparkR tests pass with R 4.1+

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.0.3, 3.2.0, 3.1.3
    • R
    • None

    Description

      Currently, there exists 6 SparkR UT failures in R 4.1.0.
      Until R 4.0.5, there was no errors.

      ══ Failed ══════════════════════════════════════════════════════════════════════
      ── 1. Failure (test_sparkSQL_arrow.R:71:3): createDataFrame/collect Arrow optimi
      collect(createDataFrame(rdf)) not equal to `expected`.
      Component “g”: 'tzone' attributes are inconsistent ('UTC' and '')
      
      ── 2. Failure (test_sparkSQL_arrow.R:143:3): dapply() Arrow optimization - type
      collect(ret) not equal to `rdf`.
      Component “b”: 'tzone' attributes are inconsistent ('UTC' and '')
      
      ── 3. Failure (test_sparkSQL_arrow.R:229:3): gapply() Arrow optimization - type
      collect(ret) not equal to `rdf`.
      Component “b”: 'tzone' attributes are inconsistent ('UTC' and '')
      
      ── 4. Error (test_sparkSQL.R:1454:3): column functions ─────────────────────────
      Error: (converted from warning) cannot xtfrm data frames
      Backtrace:
        1. base::sort(collect(distinct(select(df, input_file_name())))) test_sparkSQL.R:1454:2
        2. base::sort.default(collect(distinct(select(df, input_file_name()))))
        5. base::order(x, na.last = na.last, decreasing = decreasing)
        6. base::lapply(z, function(x) if (is.object(x)) as.vector(xtfrm(x)) else x)
        7. base:::FUN(X[[i]], ...)
       10. base::xtfrm.data.frame(x)
      
      ── 5. Failure (test_utils.R:67:3): cleanClosure on R functions ─────────────────
      `actual` not equal to `g`.
      names for current but not for target
      Length mismatch: comparison on first 0 components
      
      ── 6. Failure (test_utils.R:80:3): cleanClosure on R functions ─────────────────
      `actual` not equal to `g`.
      names for current but not for target
      Length mismatch: comparison on first 0 components
      

      Attachments

        Activity

          People

            gurwls223 Hyukjin Kwon
            dongjoon Dongjoon Hyun
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: