Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
3.0.0-alpha-1, 2.0.0
-
None
-
None
Description
In the StripeCompactPolicy#needsSingleStripeCompaction, it only checks if the file count is up to the min compaction limit.
protected boolean needsSingleStripeCompaction(StripeInformationProvider si) { int minFiles = this.config.getStripeCompactMinFiles(); for (List<HStoreFile> stripe : si.getStripes()) { if (stripe.size() >= minFiles) return true; } return false; }
When select files in stripe by ExploringCompactionPolicy#applyCompactionPolicy, there is no min size limit too.
// Sanity checks if (potentialMatchFiles.size() < minFiles) { continue; } if (potentialMatchFiles.size() > maxFiles) { continue; } // Compute the total size of files that will // have to be read if this set of files is compacted. long size = getTotalStoreSize(potentialMatchFiles); // Store the smallest set of files. This stored set of files will be used // if it looks like the algorithm is stuck. if (mightBeStuck && size < smallestSize) { smallest = potentialMatchFiles; smallestSize = size; } if (size > comConf.getMaxCompactSize(mayUseOffPeak)) { continue; } ++opts; if (size >= comConf.getMinCompactSize() && !filesInRatio(potentialMatchFiles, currentRatio)) { continue; }
While we should also limit the min size of compaction for stripes, because small stripe compactions will occupy the compaction queue.
Attachments
Issue Links
- links to