Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24465

LSHModel should support Structured Streaming for transform

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 2.3.1, 2.4.0
    • ML
    • None

    Description

      Locality Sensitive Hashing (LSH) Models (BucketedRandomProjectionLSHModel, MinHashLSHModel) are not compatible with Structured Streaming.

      This task is to add unit tests for streaming (as in SPARK-22644) for LSHModels after SPARK-12878 has been fixed.

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-12878 Dataframe fails with nested User Defined Types

          • Major - Major loss of function.
          • Resolved
          Is contained by

          Umbrella - An overarching type made of sub-tasks SPARK-21926 Compatibility between ML Transformers and Structured Streaming

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              josephkb Joseph K. Bradley
              josephkb Joseph K. Bradley
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: