Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22650

NPE in AssignmentManager (master crash on startup)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Critical
    • Resolution: Unresolved
    • 1.2.12, 1.3.5
    • None
    • master
    • Patch

    Description

      On HMaster Startup:

       

      2019-07-02 12:38:11,312 FATAL [orc3:16000.activeMasterManager] master.HMaster: Failed to become active master
      java.lang.NullPointerException
          at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
          at java.util.concurrent.ConcurrentHashMap.containsKey(ConcurrentHashMap.java:964)
          at java.util.concurrent.ConcurrentHashMap$KeySetView.contains(ConcurrentHashMap.java:4558)
          at java.util.Collections$UnmodifiableCollection.contains(Collections.java:1032)
          at org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:3094)
          at org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:495)
          at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:830)
          at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:202)
          at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1883)
          at java.lang.Thread.run(Thread.java:748)
      2019-07-02 12:38:11,312 FATAL [orc3:16000.activeMasterManager] master.HMaster: Master server abort: loaded coprocessors are: []
      2019-07-02 12:38:11,312 FATAL [orc3:16000.activeMasterManager] master.HMaster: Unhandled exception. Starting shutdown.
      java.lang.NullPointerException
          at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
          at java.util.concurrent.ConcurrentHashMap.containsKey(ConcurrentHashMap.java:964)
          at java.util.concurrent.ConcurrentHashMap$KeySetView.contains(ConcurrentHashMap.java:4558)
          at java.util.Collections$UnmodifiableCollection.contains(Collections.java:1032)
          at org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:3094)
          at org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:495)
          at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:830)
          at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:202)
          at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1883)
          at java.lang.Thread.run(Thread.java:748)

      It happens when regionLocation is null, which may happen just above on line 3086 (or as returned by getRegionServer)

      We had this on 1.2.12 with the corresponding patch, but since it is not supported anymore, did not submit it.

      Attached is the patch for 1.3.5. Did not test it in 1.4+

       

       

       

      Attachments

        1. HBASE-22650.branch-1.001.patch
          1 kB
          Sylvain Veyrié
        2. AssignmentManager-NPE.patch
          1 kB
          Sylvain Veyrié

        Activity

          People

            Unassigned Unassigned
            sveyrie Sylvain Veyrié
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: