Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17861

Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 1.4.0, 1.3.3, 1.2.7
    • None
    • Reviewed
    • Some object store does not support unix style permission, this jira fix the permission check issue when specify staging dir in different file system. Currently it covers s3, wasb, swift.

    Description

      Found some issue, when set up HBASE-17437: Support specifying a WAL directory outside of the root directory.

      The region server are showdown when I add following config into hbase-site.xml
      hbase.rootdir = s3a://xx//xx
      hbase.wal.dir = hdfs://xx/xx
      hbase.coprocessor.region.classes = org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
      Error is below

      org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw java.lang.IllegalStateException: Directory already exists but permissions aren't set to '-rwx--x--x'
      

      The reason is that, when hbase enable securebulkload, hbase will create a folder in s3, it can not set above permission, because in s3, all files are listed as having full read/write permissions and all directories appear to have full rwx permissions. See Object stores have differerent authorization models in https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html

      Attachments

        1. HBASE-17861-V1.patch
          2 kB
          Yi Liang
        2. HBASE-17861.branch-1.V1.patch
          2 kB
          Yi Liang
        3. HBASE-17861.branch-1.V2.patch
          2 kB
          Yi Liang
        4. HBASE-17861.branch-1.V3.patch
          3 kB
          Yi Liang
        5. HBASE-17861.branch-1.V4.patch
          3 kB
          Yi Liang

        Issue Links

          Activity

            People

              easyliangjob Yi Liang
              easyliangjob Yi Liang
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: