Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-17460

Make All HDFS Server side component uniform use the value of `dfs.internal.nameservices` to override `dfs.nameservices`

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • hdfs, rbf
    • None

    Description

        In HDFS, `dfs.internal.nameservices` is a server-side configuration, while `dfs.nameservices` is both a server-side and HDFSClient-side configuration.
       
        The internal configuration implies it's for internal systems. 
      Now on the DataNode and JournalNode side, the priority of dfs.internal.nameservices configuration is higher than `dfs.nameservices`, which makes sense. However, on the ZKFC + NameNode side, internal configuration isn't considered.
       
        In such a scenario, when I enable RBF, the NameNode side expects `dfs.nameservices`=ns1,ns2 (It now ignoring `dfs.internal.nameservices`),  while the HDFSClient side expects `dfs.nameservices`=ns1,ns2,nsRouter (It now ignoring `dfs.internal.nameservices`). 
      This means they can't use the same configuration, It would be too inconvenient

       
        If I unify all server-side configurations to use `dfs.internal.nameservices`, meaning using the value of `dfs.internal.nameservices` to override `dfs.nameservices` during process startup,
      it brings the following benefits:
        1、Both server and HDFSClient can use the same configuration: `dfs.nameservices`=ns1,ns2,nsRouter, `dfs.internal.nameservices`=ns1,ns2.
        2、Previously, the priority of `dfs.internal.nameservices` over `dfs.nameservices` was only considered in DataNode and JournalNode configurations,  but NameNode didn't consider dfs.internal.nameservices. This inconsistency in internal services doesn't look good and can be avoided. And now, by unifying them, it better aligns with the original semantic meaning of the word "internal" as it pertains to internal systems.

      Attachments

        Activity

          People

            Unassigned Unassigned
            bigdata_zoodev xiaojunxiang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: