Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6009

Tools based on favored node feature for isolation

    XMLWordPrintableJSON

Details

    • Task
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 2.3.0
    • None
    • None
    • None

    Description

      There're scenarios like mentioned in HBASE-6721 and HBASE-4210 that in multi-tenant deployments of HBase we prefer to specify several groups of regionservers to serve different applications, to achieve some kind of isolation or resource allocation. However, although the regionservers are grouped, the datanodes which store the data are not, which leads to the case that one datanode failure affects multiple applications, as we already observed in our product environment.

      To relieve the above issue, we could take usage of the favored node feature (HDFS-2576) to make regionserver able to locate data within its group, or say make datanodes also grouped (passively), to form some level of isolation.

      In this case, or any other case that needs datanodes to group, we would need a bunch of tools to maintain the "group", including:
      1. Making balancer able to balance data among specified servers, rather than the whole set
      2. Set balance bandwidth for specified servers, rather than the whole set
      3. Some tool to check whether the block is "cross-group" placed, and move it back if so

      This JIRA is an umbrella for the above tools.

      Attachments

        Issue Links

          Activity

            People

              liyu Yu Li
              liyu Yu Li
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated: