Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27305

add an option to skip file splitting when bulkload hfiles

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0-alpha-3
    • 2.5.0, 3.0.0-alpha-4
    • tooling
    • None
    • Reviewed
    • Add a 'hbase.loadincremental.fail.if.need.split.hfile' configuration. If set to true, th bulk load operation will fail immediately if we need to split the hfiles. This can be used to prevent unexpected time consuming bulk load operation.

    Description

      When bulkload hfiles, if the key range of the hfile does not match the key range of the region, the BulkLoadHFilesTool will split hfile to fit make the key range of the new file match the key range of the region. If there are many files to be split, the load on the BulkLoadHFilesTool will be very high. Sometimes we want to avoid this situation, just directly fail and regenerate new hfiles. Here we try to introduce a new option, When the above problem is encountered, an exception will be thrown and let the upper client handle it.

      Attachments

        Issue Links

          Activity

            People

              frostruan ruanhui
              frostruan ruanhui
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: