Uploaded image for project: 'REEF (Retired)'
  1. REEF (Retired)
  2. REEF-1776

Use YARN proxy user to register REEF Driver in Unmanaged AM mode

    XMLWordPrintableJSON

Details

    Description

      When launching REEF application in Unmanaged AM mode from another YARN application, two Drivers access the same global object UserGroupInformation from YARN API. That issue prevents us from running REEF applications on YARN in REEF-on-REEF or REEF-on-Spark settings. We have to implement a mechanism to create proxy YARN users and register Unmanaged AM Drivers in that proxy user context and update the API so that REEF client can pass the user credentials into the driver setup.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue REEF-1667 REEF-on-REEF example

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. REEF-1747 Workaround to allow two AMs in one YARN container

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Sub-task - The sub-task of the issue YARN-5531 UnmanagedAM pool manager for federating application across clusters

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              motus Sergiy Matusevych
              motus Sergiy Matusevych
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 168h
                  168h
                  Remaining:
                  Remaining Estimate - 168h
                  168h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified