Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20769

getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0, 1.4.0, 2.0.0
    • 3.0.0-alpha-1, 2.1.0, 1.3.3, 1.4.6, 2.0.2
    • None
    • None
    • Reviewed

    Description

      When numSplits > 1, getSplits may create split that has start row smaller than user specified scan's start row or stop row larger than user specified scan's stop row.

      
              byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, true);
              for (int i = 0; i < sp.length - 1; i++) {
                if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), scan.getStopRow(), sp[i],
                        sp[i + 1])) {
                  List<String> hosts =
                      calculateLocationsForInputSplit(conf, htd, hri, tableDir, localityEnabled);
      
                  Scan boundedScan = new Scan(scan);
                  boundedScan.setStartRow(sp[i]);
                  boundedScan.setStopRow(sp[i + 1]);
      
                  splits.add(new InputSplit(htd, hri, hosts, boundedScan, restoreDir));
                }
              }
      
      

      Since we split keys by the range of regions, when sp[i] < scan.getStartRow() or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range that over user defined scan.

      fix should be simple:

      
      boundedScan.setStartRow(
       Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
       boundedScan.setStopRow(
       Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i + 1]);
      
      

      I will also try to add UTs to help discover this problem

      Attachments

        1. HBASE-20769.master.004.patch
          8 kB
          Jingyun Tian
        2. HBASE-20769.master.003.patch
          6 kB
          Jingyun Tian
        3. HBASE-20769.master.002.patch
          6 kB
          Jingyun Tian
        4. HBASE-20769.master.001.patch
          6 kB
          Jingyun Tian
        5. HBASE-20769.branch-1.3.001.patch
          7 kB
          Jingyun Tian
        6. HBASE-20769.branch-1.001.patch
          7 kB
          Jingyun Tian

        Issue Links

          Activity

            People

              tianjingyun Jingyun Tian
              tianjingyun Jingyun Tian
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: