Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30602 SPIP: Support push-based shuffle to improve shuffle efficiency
  3. SPARK-36266

Rename classes in shuffle RPC used for block push operations

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 3.2.0
    • Spark Core
    • None

    Description

      In the current implementation of push-based shuffle, we are reusing certain code between both block fetch and block push.

      This is generally good except that certain classes that are meant to be used for both block fetch and block push now have names that indicate they are only for block fetches, which is confusing.

      This ticket renames these classes to be more generic to be reused across both block fetch and block push.

      Attachments

        Activity

          People

            mshen Min Shen
            mshen Min Shen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: