Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.6.0, 3.0.0-alpha-3, 2.4.15, 2.5.2
-
None
-
None
Description
Besides the fix for HStore.getScanners for FNFE on StoreFileScanner after a flush followed by a compaction in HBASE-27484, there is a another case would cause FNFE:
- When StoreScanner is scanning, there is a flush followed by a compaction. When the flushed is completed in HStore.completeFlush and after it adds the new HStoreFile in StoreEngine, it would notify the StoreScanner.updateReaders by HStore.notifyChangedReadersObservers with the new flushed HStoreFile:
private boolean completeFlush(List<HStoreFile> sfs, long snapshotId) throws IOException { // NOTE:we should keep clearSnapshot method inside the write lock because clearSnapshot may // close {@link DefaultMemStore#snapshot}, which may be used by // {@link DefaultMemStore#getScanners}. storeEngine.addStoreFiles(sfs, snapshotId > 0 ? () -> this.memstore.clearSnapshot(snapshotId) : () -> { }); // notify to be called here - only in case of flushes notifyChangedReadersObservers(sfs); if (LOG.isTraceEnabled()) { long totalSize = getTotalSize(sfs); String traceMessage = "FLUSH time,count,size,store size,store files [" + EnvironmentEdgeManager.currentTime() + "," + sfs.size() + "," + totalSize + "," + storeSize + "," + storeEngine.getStoreFileManager().getStorefileCount() + "]"; LOG.trace(traceMessage); } return needsCompaction(); }
- But if before entering StoreScanner.updateReaders , the compaction task starts and it compacts the new flushed HStoreFile and then HStore.closeAndArchiveCompactedFiles is called by CompactedHFilesDischarger to delete the HStoreFile.
- When StoreScanner.updateReaders continues to execute, it would read already deleted HStoreFile and throws FNFE.
Attachments
Issue Links
- relates to
-
HBASE-27484 FNFE on StoreFileScanner after a flush followed by a compaction
- Resolved
- links to