Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21575

memstore above high watermark message is logged too much

    XMLWordPrintableJSON

Details

    Description

      100s of Mb of logs like this, in a tight loop:

      2018-12-08 10:27:00,462 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3646ms
      2018-12-08 10:27:00,463 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3647ms
      2018-12-08 10:27:00,463 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3647ms
      2018-12-08 10:27:00,464 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3648ms
      2018-12-08 10:27:00,464 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3648ms
      2018-12-08 10:27:00,465 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3649ms
      2018-12-08 10:27:00,465 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3649ms
      2018-12-08 10:27:00,466 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3650ms
      2018-12-08 10:27:00,466 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3650ms
      2018-12-08 10:27:00,467 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3651ms
      2018-12-08 10:27:00,469 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3653ms
      2018-12-08 10:27:00,470 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3654ms
      2018-12-08 10:27:00,470 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3654ms
      2018-12-08 10:27:00,471 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3655ms
      2018-12-08 10:27:00,471 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3655ms
      2018-12-08 10:27:00,472 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3656ms
      2018-12-08 10:27:00,472 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3656ms
      2018-12-08 10:27:00,473 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3657ms
      2018-12-08 10:27:00,474 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3658ms
      2018-12-08 10:27:00,475 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3659ms
      2018-12-08 10:27:00,476 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3660ms
      2018-12-08 10:27:00,476 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3660ms
      2018-12-08 10:27:00,477 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3661ms
      2018-12-08 10:27:00,477 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3661ms
      2018-12-08 10:27:00,478 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3662ms
      2018-12-08 10:27:00,479 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3663ms
      2018-12-08 10:27:00,479 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3663ms
      2018-12-08 10:27:00,480 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3664ms
      2018-12-08 10:27:00,480 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3664ms
      2018-12-08 10:27:00,481 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3665ms
      2018-12-08 10:27:00,482 WARN  [RpcServer.default.FPBQ.Fifo.handler=2,queue=2,port=17020] regionserver.MemStoreFlusher: Memstore is above high water mark and block 3666ms
      
      

      Attachments

        1. HBASE-21575.patch
          1 kB
          Sergey Shelukhin
        2. HBASE-21575.01.patch
          1 kB
          Sergey Shelukhin

        Activity

          People

            sershe Sergey Shelukhin
            sershe Sergey Shelukhin
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: