Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13522

HDFS-13522: Add federated nameservices states to client protocol and propagate it between routers and clients.

    XMLWordPrintableJSON

Details

    Description

      Changes will need to occur to the router to support the new observer node.

      One such change will be to make the router understand the observer state, e.g. FederationNamenodeServiceState.

      This patch captures the state of all namespaces in the routers and propagates it to clients. A follow up patch will change router behavior to direct requests to the observer.

       

       

      Attachments

        1. HDFS-13522_WIP.patch
          80 kB
          Hemanth Boyina
        2. HDFS-13522.001.patch
          52 kB
          Surendra Singh Lilhore
        3. HDFS-13522.002.patch
          93 kB
          zhuobin zheng
        4. observer_reads_in_rbf_proposal_simbadzina_v1.pdf
          171 kB
          Simbarashe Dzinamarira
        5. observer_reads_in_rbf_proposal_simbadzina_v2.pdf
          320 kB
          Simbarashe Dzinamarira
        6. rbf_observer_reads - read_own_writes.pdf
          34 kB
          Simbarashe Dzinamarira
        7. rbf_observer_reads - third_party_reads.pdf
          56 kB
          Simbarashe Dzinamarira
        8. RBF_ Observer support.pdf
          298 kB
          CR Hota
        9. Router+Observer RPC clogging.png
          101 kB
          CR Hota
        10. ShortTerm-Routers+Observer.png
          155 kB
          CR Hota

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16826 [RBF SBN] ConnectionManager should advance the client stateId for every request

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16831 [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16837 [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace

          • Major - Major loss of function.
          • Resolved
          Dependent

          Task - A task that needs to be done. HDFS-16589 RBF: Support cluster level SBN read switch in Router-Based Federation

          • Major - Major loss of function.
          • Open
          incorporates

          Bug - A problem which impairs or prevents the functions of the product. HDFS-14887 RBF: In Router Web UI, Observer Namenode Information displaying as Unavailable

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HDFS-13286 Add haadmin commands to transition between standby and observer

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16826 [RBF SBN] ConnectionManager should advance the client stateId for every request

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16831 [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16834 PoolAlignmentContext should not max poolLocalStateId with sharedGlobalStateId when sending requests to the namenode.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16837 [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-16827 [RBF SBN] RouterStateIdContext shouldn't update the ResponseState if client doesn't use ObserverReadProxyProvider

          • Major - Major loss of function.
          • Open

          Task - A task that needs to be done. HDFS-16889 Backport JIRAs related to RBF SBN to branch-3.3

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16821 Fix regression in HDFS-13522 that enables observer reads by default.

          • Major - Major loss of function.
          • Resolved
          requires

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-18345 Enhance client protocol to propagate last seen state IDs for multiple nameservices.

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-18406 Adds alignment context to call path for creating RPC proxy with multiple connections per user.

          • Major - Major loss of function.
          • Resolved
          split to

          Sub-task - The sub-task of the issue HDFS-16767 RBF: Support observer node from Router-Based Federation

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #3005

          Web Link GitHub Pull Request #4127

          Web Link GitHub Pull Request #4311

          Web Link GitHub Pull Request #4441

          Web Link GitHub Pull Request #4523

          Web Link GitHub Pull Request #4883

          Activity

            People

              simbadzina Simbarashe Dzinamarira
              xkrogen Erik Krogen
              Votes:
              2 Vote for this issue
              Watchers:
              28 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20h 50m
                  20h 50m