Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-4714

Improve syncRequestProcessor performance

    XMLWordPrintableJSON

Details

    Description

      In the SyncRequestProcessor, a write operation is performed for each write request. Two methods are relatively time-consuming.

      1. Within SyncRequestProcessor#shouldSnapshot, the current size of the current file is retrieved, which involves a system call.

      Call stack:
      java.io.File.length(File.java)
      org.apache.zookeeper.server.persistence.FileTxnLog.getCurrentLogSize(FileTxnLog.java:211)
      org.apache.zookeeper.server.persistence.FileTxnLog.getTotalLogSize(FileTxnLog.java:221)
      org.apache.zookeeper.server.persistence.FileTxnSnapLog.getTotalLogSize(FileTxnSnapLog.java:671)
      org.apache.zookeeper.server.ZKDatabase.getTxnSize(ZKDatabase.java:790)
      org.apache.zookeeper.server.SyncRequestProcessor.shouldSnapshot(SyncRequestProcessor.java:145)
      org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:182)

      2. Within ZKDatabase#append, the current position of the current file is retrieved, which also involves a system call.

      Call stack:
      sun.nio.ch.FileDispatcherImpl.seek(FileDispatcherImpl.java)
      sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:264)
      org.apache.zookeeper.server.persistence.FilePadding.padFile(FilePadding.java:76)
      org.apache.zookeeper.server.persistence.FileTxnLog.append(FileTxnLog.java:298)
      org.apache.zookeeper.server.persistence.FileTxnSnapLog.append(FileTxnSnapLog.java:592)
      org.apache.zookeeper.server.ZKDatabase.append(ZKDatabase.java:678)
      org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:181)

      Therefore, it is best to maintain the current size and position of the current file ourselves, as this can greatly improve performance.

      Attachments

        1. 761688051587_.pic.jpg
          161 kB
          Yan Zhao

        Issue Links

          Activity

            People

              tison Zili Chen
              horizonzy Yan Zhao
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 10m
                  3h 10m