Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27495

Improve HFileLinkCleaner to validate back reference links ahead the next traverse

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.6.0, 3.0.0-alpha-4, 2.5.2
    • 2.6.0, 3.0.0-alpha-4, 2.5.2
    • master
    • None
    • Reviewed

    Description

      We found a a race in the CleanerChore related to back reference links. When the HFileLinkCleaner runs for a file it can make 2 decisions depending on the file types.

      • HFiles, The cleaner for HFile deletion only checks if the .links-<> directory is present with files. 
      • Back reference links, the cleaner checks if the forward link is still available in the data directory.

      The logic and order how the cleaner checks these 2 files matters. When the back reference is checked first it can remove both the reference and the HFile from the archive, however, when it first runs for the HFile then only the back-reference is removed. In this case, the HFile is only deleted in the next iteration of the CleanerChore, and it could be very slow if the list of files are huge in case of using object store.

      The goal of this task is to improve traverse of the archived HFile, reusing the list of found back reference files, and immediately apply the checks for the Back reference links.

      Attachments

        Issue Links

          Activity

            People

              taklwu Tak-Lon (Stephen) Wu
              taklwu Tak-Lon (Stephen) Wu
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: