Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3787

[Gridmix] Improve STRESS mode

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      JobMonitor can now deploy multiple threads for faster job-status polling. Use 'gridmix.job-monitor.thread-count' to set the number of threads. Stress mode now relies on the updates from the job monitor instead of polling for job status. Failures in job submission now get reported to the statistics module and ultimately reported to the user via summary.
      Show
      JobMonitor can now deploy multiple threads for faster job-status polling. Use 'gridmix.job-monitor.thread-count' to set the number of threads. Stress mode now relies on the updates from the job monitor instead of polling for job status. Failures in job submission now get reported to the statistics module and ultimately reported to the user via summary.
    • gridmix3 stress

    Description

      Gridmix STRESS mode can be improved as follows:
      1. The sleep time in JobMonitor can be reduced and/or made configurable
      2. Map and reduce load calculation in StressJobFactory can be done in one loop
      3. Updating the overload status from the job submitter thread (inline)
      4. Optimizations to avoid un-necessary progress check (which inturn would result into delay)

      Attachments

        Issue Links

        relates to

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-3769 [Gridmix] Improve the way job monitor maintains running jobs

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amar_kamat Amar Kamat Assign to me
            amar_kamat Amar Kamat
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment