Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6648

Order of namenodes in ConfiguredFailoverProxyProvider is undefined

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.0
    • 2.9.0, 3.0.0-alpha4
    • ha, hdfs-client
    • None
    • Reviewed

    Description

      In org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider, in the constructor, there's a map <nameservice : < service-id : service-rpc-address > > (DFSUtil.getHaNnRpcAddresses). It's a LinkedHashMap of HashMaps. The order is kept for nameservices. Then to find active namenode, for nameservice, we get HashMap of <service-id : service-rpc-address > for requested nameservice (taken from URI request), And for this HashMap we get values - order of this collection is not strictly defined! In the code: 

      Collection<InetSocketAddress> addressesOfNns = addressesInNN.values();

      And then we put these values (in not defined order) into ArrayList of proxies, and then in getProxy we start from first proxy in the list and failover to next if needed.

      It would make sense for ConfiguredFailoverProxyProvider to keep order of proxies/namenodes defined in hdfs-site.xml.

      Attachments

        1. HDFS-6648-001.patch
          4 kB
          Íñigo Goiri
        2. HDFS-6648-000.patch
          2 kB
          Íñigo Goiri

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. HDFS-6440 Support more than 2 NameNodes

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HDFS-12976 Introduce ObserverReadProxyProvider

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDFS-10467 Router-based HDFS federation

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13653 Make dfs.client.failover.random.order a per nameservice configuration

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-15350 Set dfs.client.failover.random.order to true as default

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              elgoiri Íñigo Goiri
              ravwojdyla Rafal Wojdyla
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: