Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12253

ViewFileSystem getFileStatus java.lang.ArrayIndexOutOfBoundsException: 0

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.6.0
    • None
    • fs
    • None
    • hadoop 2.6.0 hive 1.1.0 tez0.7 cenos6.4

    Description

      sWhen I enable hdfs federation.I run a query on hive on tez. Then it occur a exception:

      8.784 PM	WARN	org.apache.hadoop.security.UserGroupInformation	No groups available for user tangjijun
      3:12:28.784 PM	ERROR	org.apache.hadoop.hive.ql.exec.Task	Failed to execute tez graph.
      
      java.lang.ArrayIndexOutOfBoundsException: 0
      	at org.apache.hadoop.fs.viewfs.ViewFileSystem$InternalDirOfViewFs.getFileStatus(ViewFileSystem.java:771)
      	at org.apache.hadoop.fs.viewfs.ViewFileSystem.getFileStatus(ViewFileSystem.java:359)
      	at org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:955)
      	at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:184)
      	at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:787)
      	at org.apache.tez.client.TezClient.start(TezClient.java:337)
      	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:191)
      	at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:234)
      	at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136)
      	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
      	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1640)
      	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1399)
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)
      	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144)
      	at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69)
      	at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
      	at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      

      I digging into the issue,I found the code snippet in ViewFileSystem.java as follows:

       @Override
          public FileStatus getFileStatus(Path f) throws IOException {
            checkPathIsSlash(f);
            return new FileStatus(0, true, 0, 0, creationTime, creationTime,
                PERMISSION_555, ugi.getUserName(), ugi.getGroupNames()[0],
      
                new Path(theInternalDir.fullPath).makeQualified(
                    myUri, ROOT_PATH));
          }
      

      If the node in cluster haven't creat user like tangjijun,ugi.getGroupNames()[0] will throw ArrayIndexOutOfBoundsException.Because no user mean no group.

      I create user tangjijun on that node. Then the job was executed normally.
      I think this code should check ugi.getGroupNames() is empty.When it is empty ,then print some log. Not to throw exception.

      Attachments

        1. HADOOP-12253.2.patch
          4 kB
          Ajith S
        2. HADOOP-12253.patch
          3 kB
          Ajith S

        Issue Links

          Activity

            People

              ajithshetty Ajith S
              r7raul JiJun Tang
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: