Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8827

Support Spark 3 on Spark Structured Streaming Runner

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • runner-spark

    Description

      Spark Structured Streaming Runner uses currently non stable Spark APIs for its Source translation, the DataSourceV2 API was changed on Spark 3. We need probably a new class or a new compatibility layer to support this for Spark 3.

      Attachments

        Activity

          People

            Unassigned Unassigned
            iemejia Ismaël Mejía
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: