Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-4755 HBase based block placement in DFS
  3. HBASE-7932

Do the necessary plumbing for the region locations in META table and send the info to the RegionServers

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 0.95.1
    • None
    • None
    • Hide
      Introduces "favored nodes" for region files. Has an implementation of LoadBalancer that assigns favored nodes for each region. There is a Primary RegionServer that hosts the region, and then there is Secondary and Tertiary RegionServers. Currently, the favored nodes information is used in creating HDFS files - the Primary RegionServer passes the primary, secondary, tertiary node addresses as hints to the DistributedFileSystem API for creating files on the filesystem. These nodes are treated as hints by the HDFS to place the blocks of the file (this will be complete when HBASE-7942 is addressed). This alleviates the problem to do with reading from remote nodes (since we can make the Secondary RegionServer as the new Primary RegionServer) after a region is recovered. This should help provide consistent read latencies for the regions even when their primary region servers die. The implementation of the balancer is not complete as of this issue.
      Show
      Introduces "favored nodes" for region files. Has an implementation of LoadBalancer that assigns favored nodes for each region. There is a Primary RegionServer that hosts the region, and then there is Secondary and Tertiary RegionServers. Currently, the favored nodes information is used in creating HDFS files - the Primary RegionServer passes the primary, secondary, tertiary node addresses as hints to the DistributedFileSystem API for creating files on the filesystem. These nodes are treated as hints by the HDFS to place the blocks of the file (this will be complete when HBASE-7942 is addressed). This alleviates the problem to do with reading from remote nodes (since we can make the Secondary RegionServer as the new Primary RegionServer) after a region is recovered. This should help provide consistent read latencies for the regions even when their primary region servers die. The implementation of the balancer is not complete as of this issue.
    • 0.96notable

    Description

      To be specific, this jira is to do the following during createTable:
      1. Favored node calculation
      2. Persisting of calculation in .META.
      3. Letting RegionServers know of the favored nodes for new regions
      4. Addition of a balancer that will honor favored nodes but it is currently a noop.

      Attachments

        1. 7932-1.patch
          124 kB
          Devaraj Das
        2. 7932-3.patch
          116 kB
          Devaraj Das
        3. 7932-8.1.patch
          130 kB
          Devaraj Das
        4. 7932-8.patch
          132 kB
          Devaraj Das
        5. 7932-simple-2.patch
          38 kB
          Devaraj Das
        6. 7932-simple-with-meta-pb.1.txt
          138 kB
          Devaraj Das
        7. 7932-simple-with-meta-pb.2.txt
          138 kB
          Devaraj Das
        8. 7932-simple-with-meta-pb.6.txt
          155 kB
          Devaraj Das
        9. 7932-wip.patch
          123 kB
          Devaraj Das
        10. 7932-wip-2.patch
          122 kB
          Devaraj Das
        11. 7932-wip-3.patch
          124 kB
          Devaraj Das
        12. 7932-wip-4.patch
          122 kB
          Devaraj Das

        Issue Links

          Activity

            People

              ddas Devaraj Das
              ddas Devaraj Das
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: