Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3385

The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Cannot Reproduce
    • 4.3.1
    • None
    • core
    • None

    Description

      The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one . for example, I need submit a workflow by proxy user "platform" via user oozie (kerberos) , an error occur in oozie source code  WorkflowAppService.readDefinition read workflow.xml.

      2018-11-14 00:00:00,497 ERROR [CallableQueue-42]org.apache.oozie.command.wf.SubmitXCommand(517) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] XException, org.apache.oozie.command.CommandException: E0710: Could not read the workflow definition, Permission denied: user=dbzq04, access=READ, inode="/phoebus/_fileservice/users/nsplatform/platform/workflows/DataLoadWF-1427-1129/workflow.xml":platform:supergroup:rw------

      note: user  "dbzq04"  also submit some workflow at before, but current submit the workflow of user is user platform. In order to prove current user is platform , I insert some logs at oozie source code 

       

      /**  org.apache.oozie.service.HadoopAccessorService   */
      public FileSystem createFileSystem(String user, final URI uri, final Configuration   conf) throws HadoopAccessorException {
            //.........omit......
       try {
         UserGroupInformation ugi = getUGI(user);
         LOG.info("current user="+ugi);  //------ my insert log, to print proxy ugi info
         return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() {
         public FileSystem run() throws Exception {
          FileSystem fs = FileSystem.get(uri, conf);
          //------ my insert log, to print fs inner ugi info
          if(fs instanceof DistributedFileSystem){
           LOG.info("hdfs client user, "+((DistributedFileSystem)fs).getClient().toString());
          }
          return fs;
         }
       });
       }catch (InterruptedException ex) {
         throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
       }catch (IOException ex) {
         throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
       }
      }

      my log print result follows:

      2018-11-14 00:00:00,492 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] current user=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS)
      2018-11-14 00:00:00,493 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] hdfs client user, DFSClient[clientName=DFSClient_NONMAPREDUCE_-515910437_325, ugi=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS)]

       

      over above the proves Indicate at visited HDFS path of user has been altered,  where did user "dbzq04"  come from? please help me check this problem--**

      Attachments

        1. oozie-server-error.log
          55 kB
          GuangMing Lu
        2. part_source_HadoopAccessorService.txt
          1 kB
          GuangMing Lu
        3. part_source_WorkflowAppService.txt
          1 kB
          GuangMing Lu

        Activity

          People

            Unassigned Unassigned
            luguangming GuangMing Lu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: