Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.6.0, 3.0.0-beta-1
    • Balancer
    • None

    Description

      HBase uses ephemeral cache to cache the blocks by reading them from the slow storages and storing them to the bucket cache. This cache is warmed up everytime a region server is started. Depending on the data size and the configured cache size, the cache warm up can take anywhere between a few minutes to few hours. Doing this everytime the region server starts can be a very expensive process. To eliminate this, HBASE-27313 implemented the cache persistence feature where the region servers periodically persist the blocks cached in the bucket cache. This persisted information is then used to resurrect the cache in the event of a region server restart because of normal restart or crash.

      This feature aims at enhancing this capability of HBase to enable the balancer implementation considers the cache allocation of each region on region servers when calculating a new assignment plan and uses the region/region server cache allocation info reported by region servers which takes into account to calculate the percentage of HFiles cached for each region on the hosting server, and then use that as another factor when deciding on an optimal, new assignment plan.

       

      A design document describing the balancer can be found at https://docs.google.com/document/d/1A8-eVeRhZjwL0hzFw9wmXl8cGP4BFomSlohX2QcaFg4/edit?usp=sharing

      Attachments

        Activity

          People

            ragarkar Rahul Agarkar
            ragarkar Rahul Agarkar
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: