Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4587

Support fetch by key boundaries for memcmp types

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • nodemanager, task
    • None

    Description

      Intermediate data addressable by key support not only restartable streams, but partitioning after the map output are written. With sampling of map output, a job can implement a total-order and tune the number of reduces around skew. It is possible to implement something similar for non-memcmp types, but it is significantly more complex.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            cdouglas Christopher Douglas

            Dates

              Created:
              Updated:

              Slack

                Issue deployment