Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31017

Test for shuffle requests packaging with different size and numBlocks limit

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Spark Core, Tests
    • None

    Description

      When packaging shuffle fetch requests in ShuffleBlockFetcherIterator, there are two limitations: maxBytesInFlight and maxBlocksInFlightPerAddress. However, we don’t have test cases to test them both, e.g. the size limitation is hit before the numBlocks limitation.

      We should add test cases in ShuffleBlockFetcherIteratorSuite to test:

      1. the size limitation is hit before the numBlocks limitation
      1. the numBlocks limitation is hit before the size limitation

      Attachments

        Issue Links

          Activity

            People

              Ngone51 wuyi
              Ngone51 wuyi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: