Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-17026

RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • rbf
    • Reviewed

    Description

      TheĀ NamenodeHeartbeatService currently calls each of the Namenode's JMX endpoint every time it wakes up (default value is every 5 seconds).

      In a cluster with 40 routers, we have observed service degradation on some of theĀ  Namenodes, since the JMX request obtains Datanode status and blocks other RPC requests. However, JMX report data doesn't seem to be used for critical paths on the routers.

      We should configure the NamenodeHeartbeatService so it updates the JMX reports on a slower frequency than the Namenode states or to disable the reports completely.

      The class calls out the JMX request being optional even though there is no implementation to turn it off:

      // Read the stats from JMX (optional)
      updateJMXParameters(webAddress, report);

      Attachments

        1. HDFS-17026-branch-3.3.patch
          14 kB
          Hector Sandoval Chaverri

        Activity

          People

            hchaverri Hector Sandoval Chaverri
            hchaverri Hector Sandoval Chaverri
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: