Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5441

Pig skew join tez grace reducer fails to find shuffle data

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.17.0
    • 0.18.0
    • tez
    • None
    • Reviewed

    Description

      User pig tez skew join encountered issue of not finding shuffle data from the sampler aggregate vertex. The right side join has >1 reducers.

      For workaround adjust tez.runtime.transfer.data-via-events.max-size to avoid spill to disk for the sampler aggregation vertex. 

       

      Attachments

        1. PIG-5441.patch
          1 kB
          Yi Zhang

        Issue Links

          Activity

            People

              yigress Yi Zhang
              yigress Yi Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: