XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.1
    • 3.3.0, 3.2.2
    • Structured Streaming
    • None

    Description

      Implement option to skip null values in symmetric hash impl of stream-stream joins

      • In the symmetric has join state manager, we can receive entries with null values for a key and that caused the `removeByValue` and get iterators to fail and run into the NullPointerException.
      • This is possible if the state recovered is written from a old spark version or its corrupted on disk. Since we don't have a utility to query this state, we would like to provide a conf option to skip nulls for the symmetric hash impl in stream stream joins.

       

      Attachments

        Activity

          People

            anishshri-db Anish Shrigondekar
            anishshri-db Anish Shrigondekar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: