Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18802

Collision of config key name fs.viewfs.mounttable.default.name.key to other keys that specify the entry point to mount tables

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • None
    • None
    • common, conf, fs

    Description

      What happened:

      When manually set fs.viewfs.mounttable.default.name.key to default (the same as default value) in HCommon, test org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches would fail.
      But the test can pass if this parameter is not manually set in the configuration file.

      Where's the bug:

      In the constructor of InodeTree, the tree attempts to get all the mount table entry points set by user in the configuration and process them one by one:

      for (Entry<String, String> si : config) {       
          final String key = si.getKey();       
          if (!key.startsWith(mountTablePrefix)) {         
              continue;       
          }  
          
          gotMountTableEntry = true;       
          LinkType linkType;       
          String src = key.substring(mountTablePrefix.length());       
          ...
      

      Here mountTablePrefix="fs.viewfs.mounttable.default.". However, it just so happens that the name of the configuration users use to specify the default mount table is fs.viewfs.mounttable.default.name.key. Thus, if a user specifies the default mount table and uses InodeTree the name.key would be falsely parsed as the entry point to one of the mount tables, which would cause InodeTree to throw an exception since name.key is not a valid entry.

      Stack trace:

      java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize: Invalid entry in Mount table in config: name.key    
              at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
              at org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88)
              at org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38)

      How to reproduce:

      (1) Set fs.viewfs.mounttable.default.name.key to default
      (2) Run test org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches

      You can use the reproduce.sh in the attachment to easily reproduce the bug.

      We are happy to provide a patch if this issue is confirmed. 

      Attachments

        1. reproduce.sh
          0.8 kB
          ConfX

        Issue Links

          Activity

            People

              Unassigned Unassigned
              FuzzingTeam ConfX
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: