Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26171

Is Spark 2.3+ support UDT?

    XMLWordPrintableJSON

Details

    • Question
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • 2.4.0
    • None
    • Examples
    • None

    Description

      I was going through this ticket https://issues.apache.org/jira/browse/SPARK-7768?jql=text%20~%20%22public%20udt%22 and could not understand that if spark support UDT in  2.3+ version in any language (scala, python , java, R) ?

      I have class something like this

      Class Test

      {        string name;        int age; }

       

      And My UDF method is:

      public Test UDFMethod(string name, int age)

      {           Test ob = new Test();           ob.name = name;           ob.age = age; }

      Sample Spark query-   `Select *, UDFMethod(name, age) From SomeTable;`

      Now UDFMethod(name, age) will return Class Test object. So will this work in Spark SQL after using SQLUserDefinedType tag and extending UserDefinedType class. 

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            sunytyagi SUNY TYAGI
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: