Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3719

DAGImpl.computeProgress slows down dispatcher and ipc threads

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0, 0.8.5
    • None

    Description

      The Vertex readLock is held while computing progress, which prevents scheduling of tasks from happening on that vertex (which needs a writeLock).

      Making the locked section do as little work as possible would speedup the AM.

      Attachments

        1. TEZ-3719.wip.patch
          2 kB
          Rajesh Balamohan
        2. TEZ-3719.3.patch
          3 kB
          Gopal Vijayaraghavan
        3. TEZ-3719.2.patch
          3 kB
          Gopal Vijayaraghavan
        4. TEZ-3719.1.patch
          3 kB
          Gopal Vijayaraghavan
        5. tez_computeProgress_waiting_for_lock.png
          335 kB
          Rajesh Balamohan
        6. dispatcher_holding_lock.png
          376 kB
          Rajesh Balamohan

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-16635 Progressbar: Use different timeouts for running queries

          • Major - Major loss of function.
          • Closed

          Activity

            People

              gopalv Gopal Vijayaraghavan
              rajesh.balamohan Rajesh Balamohan
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: