Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1741

XInclude support broken for YARN ResourceManager

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Won't Fix
    • 2.4.0
    • None
    • resourcemanager

    Description

      The XInclude support in Hadoop configuration files (introduced via HADOOP-4944) was broken by the recent ConfigurationProvider changes to YARN ResourceManager. Specifically, YARN-1459 and, more generally, the YARN-1611 family of JIRAs for ResourceManager HA.

      The issue is that ConfigurationProvider provides a raw InputStream as a Configuration resource for what was previously a Path-based resource.

      For Path resources, the absolute file path is used as the systemId for the DocumentBuilder.parse() call:

            } else if (resource instanceof Path) {          // a file resource
      ...
                doc = parse(builder, new BufferedInputStream(
                    new FileInputStream(file)), ((Path)resource).toString());
              }
      

      The systemId is used to resolve XIncludes (among other things):

          /**
           * Parse the content of the given <code>InputStream</code> as an
           * XML document and return a new DOM Document object.
      ...
           * @param systemId Provide a base for resolving relative URIs.
      ...
           */
          public Document parse(InputStream is, String systemId)
      

      However, for loading raw InputStream resources, the systemId is set to null:

            } else if (resource instanceof InputStream) {
              doc = parse(builder, (InputStream) resource, null);
      

      causing XInclude resolution to fail.

      In our particular environment, we make extensive use of XIncludes to standardize common configuration parameters across multiple Hadoop clusters.

      Attachments

        Issue Links

          Activity

            People

              xgong Xuan Gong
              sirianni Eric Sirianni
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: