Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18484

VerifyRep by snapshot does not work when Yarn / SourceHBase / PeerHBase located in different HDFS clusters

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      As HBASE-16466 commented.

      Seems like that when source hbase cluster / peer hbase cluster / yarn cluster locate in three different HDFS cluster , it has one problem.
      when restoring the snapshot into tmpdir , we need to create region by following code (HRegion#createHRegion)

      public static HRegion createHRegion(final HRegionInfo info, final Path rootDir,
              final Configuration conf, final TableDescriptor hTableDescriptor,
              final WAL wal, final boolean initialize)
        throws IOException {
          LOG.info("creating HRegion " + info.getTable().getNameAsString()
              + " HTD == " + hTableDescriptor + " RootDir = " + rootDir +
              " Table name == " + info.getTable().getNameAsString());
          FileSystem fs = FileSystem.get(conf);                                  <-------------------  Here our code use  fs.defaultFs configuration to create region.
          Path tableDir = FSUtils.getTableDir(rootDir, info.getTable());
          HRegionFileSystem.createRegionOnFileSystem(conf, fs, tableDir, info);
          HRegion region = HRegion.newHRegion(tableDir, wal, fs, conf, info, hTableDescriptor, null);
          if (initialize) region.initialize(null);
          return region;
        }
      

      When source cluster & peer cluster locate in two difference file systems , then their fs.defaultFs should be difference, so at least one cluster will fail when restore snapshot into tmpdir . after I added the following fix, it works fine for me.

      -FileSystem fs = FileSystem.get(conf);  
      +FileSystem fs = rootDir.getFileSystem(conf);
      

      Attachments

        1. HBASE-18484.v1.patch
          1 kB
          Zheng Hu
        2. HBASE-18484.v2.patch
          13 kB
          Zheng Hu
        3. HBASE-18484.v3.patch
          13 kB
          Zheng Hu

        Issue Links

          Activity

            People

              openinx Zheng Hu
              openinx Zheng Hu
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: