Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-16842

DatanodeHttpServer cannot get the new configuration of the new nameservice even after refreshNamenodes is run

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.2
    • 3.1.2
    • datanode

    Description

      'refreshNamenodes' should be run when a new nameservice is added.
      However, a 'DatanodeHttpServer' cannot get the new configuration of the new nameservice even after 'refreshNamenodes' is run.
      Therefore, when a client tries to access the new namespace's data with a swebhdfs or REST API request, an error occurs as follows.

      Does not contain a valid host:port authority: NEW_NAMESPACE

      When 'refreshNamenodes' is executed, a Datanode object reads the new configuration file and updates its configuration with 'setConf(new Configuration)'.

      public void refreshNamenodes() throws IOException {
        checkSuperuserPrivilege();
        setConf(new Configuration());
        refreshNamenodes(getConf());
      }

      On the other hand, the configuration of a DatanodeHttpServer object remains unchanged after the DatanodeHttpServer object is created.
      When a Datanode object is created, a DatanodeHttpServer is created by copying the Datanode's configuration. Since then, there's no way to update its configuration.

      public DatanodeHttpServer(final Configuration conf,
            final DataNode datanode,
            final ServerSocketChannel externalHttpChannel)
          throws IOException {
          this.restCsrfPreventionFilter = createRestCsrfPreventionFilter(conf);
          this.conf = conf;
          ...
      } 

      This causes a DatanodeHttpServer to be unable to interpret a request for the new name service.

      Therefore, a DatanodeHttpServer also needs to update its configuration to handle a request for a new name service.
      I attached a patch based on the trunk.

       

      Thanks!

      Attachments

        1. HDFS-16842.001.patch
          12 kB
          Jihye Seo

        Activity

          People

            Unassigned Unassigned
            sjh8763 Jihye Seo
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: