Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24051

When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failure

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • 2.0.1
    • None
    • None
    • None
    • Reviewed

    Description

      When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log:

      regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95)

      regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95)

      [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms
      GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms
      GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms

      In particular, GC pauses occurred 283 times in half an hour.

      The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer

      See HDFS-14308,but after the fix, hbase did not return to normal.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shenshengli shenshengli Assign to me
            shenshengli shenshengli

            Dates

              Created:
              Updated:

              Slack

                Issue deployment