Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13610

[Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      See HDFS-13150 for full design.

      This JIRA is targeted at cleanup tasks:

      • Add in integration testing. We can expand TestStandbyInProgressTail
      • Documentation in HDFSHighAvailabilityWithQJM
      • Remove the dummy sync added as part of HDFS-10519; it is unnecessary since now in-progress tailing does not rely on the JN committedTxnId

      A few bugs are also fixed:

      • Due to changes in HDFS-13609 to enable use of the RPC mechanism whenever inProgressOK is true, there were codepaths which would use the RPC mechanism even when dfs.ha.tail-edits.in-progress was false, meaning that the JNs did not enable the cache. Update the QJM logic to only use selectRpcInputStreams if this config is true.
      • Fix a false error logged when the layout version changes
      • Fix the logging when a layout version change occurs to avoid printing out a placeholder value (Integer.MAX_VALUE)

      Attachments

        1. HDFS-13610-HDFS-12943.004.patch
          31 kB
          Erik Krogen
        2. HDFS-13610-HDFS-12943.003.patch
          25 kB
          Erik Krogen
        3. HDFS-13610-HDFS-12943.002.patch
          24 kB
          Erik Krogen
        4. HDFS-13610-HDFS-12943.001.patch
          15 kB
          Erik Krogen
        5. HDFS-13610-HDFS-12943.000.patch
          13 kB
          Erik Krogen

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            xkrogen Erik Krogen Assign to me
            xkrogen Erik Krogen
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment