Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7276

hadoop job fast fail is not working properly

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 3.2.0, 2.9.2
    • None
    • applicationmaster

    • hadoop 2.9.2, hive 2.3.2 and hive 0.13

    Description

      User operation : run a hive job with HFDS disk quota specified, and enable the fast fail feature to fast fail a job when hdfs disk quota limitation is exceeded (mapreduce.job.dfs.storage.capacity.kill-limit-exceed=true).

      Expect behaviour : the job fails fast when one task attempt fails due to exceeding the HDFS quota limitation.

      Actual behaviour : fast fail is not working at all. 

      Cause of the bug : TaskAttemptImpl does not handle fast fail at all in the FAILED_FINISHING_TRANSITION event.

       
       

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tiana528 Wang Yan Assign to me
            tiana528 Wang Yan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 0.5h
              0.5h

              Slack

                Issue deployment