Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15885

Compute StoreFile HDFS Blocks Distribution when needed

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: HFile
    • Labels:
      None

      Description

      Now when open a StoreFileReader, it always need to compute HDFS blocks distribution. But when balance a region, it will increase the region not serving time. Because it need first close region on rs A, then open it on rs B. When close region, it first preFlush, then flush the new update to a new store file. The new store file will first be flushed to tmp directory, then move it to column family directory. These need open StoreFileReader twice which means it need compute HDFS blocks distribution twice. When open region on rs B, it need open StoreFileReader and compute HDFS blocks distribution too. So when balance a region, it need compute HDFS blocks distribution three times for per new store file. This will increase the region not serving time and we don't need compute HDFS blocks distribution when close a region.

      The related three methods in HStore.
      1. validateStoreFile(...)
      2. commitFile(...)
      3. openStoreFiles(...)

        Attachments

        1. HBASE-15885.patch
          2 kB
          Guanghao Zhang

          Activity

            People

            • Assignee:
              zghao Guanghao Zhang
              Reporter:
              zghao Guanghao Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: