Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28227

Tables to which Stripe Compaction policy is applied cannot be forced to trigger Major Compaction.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.2.6
    • None
    • regionserver
    • None

    Description

          There is a table and the Stripe Compaction strategy is applied. Each region has an average value of 40G and is divided into 8 Stripes. Each Stripe is 5G. The business deletes a large amount of data. Manually triggering major compaction on the entire table and a single region does not work and cannot be selected.

          After reading the source code, the merging strategy applied under each Stripe is ExploringCompactionPolicy. This strategy has a key point. It filters the Store file list of a single Stripe. In the candidate file list, as long as there is a file that is too large in size and meets the condition, fileSize > (totalFileSize - fileSize) * (hbase.hstore.compaction.ratio default value 1.2), files will not be filtered out to participate in major compaction.

          It is necessary to support a forced merging mechanism. For scenarios where a large amount of data is deleted, or where bulkload exists, you can explicitly pass in a parameter such as foreMajor when manually triggering the major, and then perform forced Major Compaction in Stripe units to support the data. Clean up.

          

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              leo_jie longping_jie
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: