Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3678

Reduce the number of NameNode access when starting the Yarn job

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Done
    • 5.2.1
    • 5.3.0
    • None
    • None

    Description

      Before Oozie starts the Yarn application it assembles the class path which's elements' paths are resolved with

      remoteFS.resolvePath(p)

      which can cause slowness and high NameNode load.

      The intention is to make this path resolution configurable (ability to turn it on/off) via an Oozie configuration thus customers can turn it off when they experience job start slowness. Default value should result the new behaviour is active.

      Attachments

        1. OOZIE-3678-002.patch
          7 kB
          János Makai
        2. OOZIE-3678-001.patch
          7 kB
          János Makai

        Activity

          People

            jmakai János Makai
            jmakai János Makai
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: