Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24403

FsDelegationToken should cache Token

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 3.0.0-alpha-1, 2.3.0, 2.2.5
    • None
    • None
    • None

    Description

      When doing Bulkload, we find that FsDelegationToken will acquire token of NameNode everytime for a single file, although, from the comment of acquireDelegationToken(), it claims that it is trying to find token in cache firstly, but the newly requested token is never put to cache and thus the cache is still empty for the following request;

      In cases there are many files to do the bulk load, the token request will cause big short to NameNode.

       

      public void acquireDelegationToken(final FileSystem fs)
 throws IOException {
 if (userProvider.isHadoopSecurityEnabled()) {
 this.fs = fs;

 userToken = userProvider.getCurrent().getToken("HDFS_DELEGATION_TOKEN",
 fs.getCanonicalServiceName());
 if (userToken == null) {
 hasForwardedToken = false;
 try {
 userToken = fs.getDelegationToken(renewer);
 } catch (NullPointerException npe) {
 // we need to handle NullPointerException in case HADOOP-10009 is missing
 LOG.error("Failed to get token for " + renewer);
 }
 } else {
 hasForwardedToken = true;
 LOG.info("Use the existing token: " + userToken);
 }
 }
}

      Attachments

        1. 24403.patch
          12 kB
          wuchang

        Issue Links

          breaks

          Task - A task that needs to be done. HBASE-24610 TestSecureExport is failing

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #1743

          Activity

            People

              wuchang1989 wuchang
              wuchang1989 wuchang
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: