[HBASE-25921] Fix Wrong FileSystem when running `filesystem` on non-HDFS storage - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Minor
Resolution: Fixed
Affects Version/s: 1.0.0, 1.1.0, 1.2.0
Fix Version/s: hbase-operator-tools-1.2.0
Component/s: hbase-operator-tools
Labels:
None

Description

recently we found a bug that when running hbck filesystem -f on a non-HDFS and caused Wrong FS error and cannot move forward.

this change fix this problem and use FSUtils.getCurrentFileSystem to respect the root directory of what we set with hbase.rootdir in stead of just using fs.defaultFS

e.g. this is one of the captured ERROR message

Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: s3://xyz/data/default/abc, expected: hdfs://xyz/
        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:737)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:233)
        at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:1045)
        at org.apache.hadoop.hdfs.DistributedFileSystem.access$1000(DistributedFileSystem.java:131)
        at org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1112)
        at org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1109)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:1119)
        at org.apache.hadoop.hbase.util.FSUtils.listStatusWithStatusFilter(FSUtils.java:1530)
        at org.apache.hbase.hbck1.HFileCorruptionChecker.checkTableDir(HFileCorruptionChecker.java:320)
        at org.apache.hbase.hbck1.HFileCorruptionChecker.checkTables(HFileCorruptionChecker.java:431)
        at org.apache.hbase.FileSystemFsck.fsck(FileSystemFsck.java:87)
        at org.apache.hbase.HBCK2.doCommandLine(HBCK2.java:721)
        at org.apache.hbase.HBCK2.run(HBCK2.java:631)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
        at org.apache.hbase.HBCK2.main(HBCK2.java:865)