Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26970

Can't load PipelineModel that was created in Scala with Python due to missing Interaction transformer

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.0
    • 3.0.0
    • ML, PySpark
    • None

    Description

      The Interaction transformer https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/feature/Interaction.scala is missing from the set of pyspark feature transformers https://github.com/apache/spark/blob/master/python/pyspark/ml/feature.py 

      This means that it is impossible to create a model that includes an Interaction transformer with pyspark. It also means that attempting to load a PipelineModel created in Scala that includes an Interaction transformer with pyspark fails with the following error:

      AttributeError: module 'pyspark.ml.feature' has no attribute 'Interaction'
      

      Attachments

        Activity

          People

            Andrew-C Andrew Crosby
            Andrew-C Andrew Crosby
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: