Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27125

The batch size of cleaning expired mob files should have an upper bound

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.12
    • 2.5.0, 3.0.0-alpha-3, 2.4.13
    • mob
    • None
    • Configure "hbase.master.mob.cleaner.batch.size.upper.bound" to set a proper batch size of cleaning expired mob files, its default value is 10000.

    Description

      Currently the cleaning logic for expired mob files is adding all the deletable files in one directory to a list in memory and then archiving all the files in the list. But when there are millions of files need to delete, the list will be huge and make great heap memory pressure to the master.

      Attachments

        Activity

          People

            Xiaolin Ha Xiaolin Ha
            Xiaolin Ha Xiaolin Ha
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: