Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28040

sql() fails to process output of glue::glue_data()

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.4.3
    • None
    • SparkR

    Description

      glue package is quite natural for sending parameterized queries to Spark from R. Very similar to Python's format for strings. Error is as simple as

      library(glue)
      library(sparkR)
      sparkR.session()
      
      query = glue_data(list(val = 4), 'select {val}')
      sql(query)

      Error in writeType(con, serdeType) :
      Unsupported type for serialization glue
      sql(as.character(query)) works as expected but this is a bit awkward / post-hoc

      Attachments

        Activity

          People

            Unassigned Unassigned
            michaelchirico Michael Chirico
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: