Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-575

Support Async Compaction for spark streaming writes to hudi table

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 0.7.0
    • spark

    Description

      Currenlty, only inline compaction is supported for Structured streaming writes. 

       

      We need to 

      • Enable configuring async compaction for streaming writes 
      • Implement a parallel compaction process like we did for delta streamer

      Attachments

        Issue Links

          Activity

            People

              vbalaji Balaji Varadarajan
              vbalaji Balaji Varadarajan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: