Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-26249

Ameliorate compaction made by bulk-loading files

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.5.0, 3.0.0-alpha-2
    • None
    • None

    Description

      HBASE-25213 introduced to trigger compact after bulkload, but if a store needs compaction, each file loaded to this store will add a compaction runner of this store to the compaction queue. Then repeatedly compute for needCompaction and selectFiles for this store, and the compaction of the store may be continuous.

      We can mark the queued compaction runners for the stores, and avoid repeatedly adding same store compactions to the queue after bulk load files. This method can reduce the pressure during the period just after bulk loading and spread the pressure to other times by the periodic CompactionChecker.

      For the need of splitting mentioned before, after the completion of compaction, it will trigger split of the region if it needs. So we do not need to trigger split after bulk loading.

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Xiaolin Ha Xiaolin Ha Assign to me
            Xiaolin Ha Xiaolin Ha
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment