Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4147

Reduce NN calls in RecoveryService::handleRecoveryEvent

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • None
    • 0.10.0, 0.9.3
    • None
    • None

    Description

      https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/history/recovery/RecoveryService.java#L459

      https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/history/recovery/RecoveryService.java#L467

      create(overwrite=false) already checks for file exists and throws relevant exception. It should be possible to reduce recoveryDirFS.exists(dagFilePath) check to help in concurrent job runs (e.g 50-100 AM trying to run very short jobs).

      Attachments

        1. TEZ-4147.01.patch
          2 kB
          László Bodor

        Activity

          People

            abstractdog László Bodor
            rajesh.balamohan Rajesh Balamohan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: