Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18055

Async Profiler endpoint for Hadoop daemons

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • common
    • Reviewed

    Description

      Async profiler (https://github.com/jvm-profiling-tools/async-profiler) is a low overhead sampling profiler for Java that does not suffer from Safepoint bias problem. It features HotSpot-specific APIs to collect stack traces and to track memory allocations. The profiler works with OpenJDK, Oracle JDK and other Java runtimes based on the HotSpot JVM.

      Async profiler can also profile heap allocations, lock contention, and HW performance counters in addition to CPU.

      We have an httpserver based servlet stack hence we can use HIVE-20202 as an implementation template to provide async profiler as servlet for Hadoop daemons. Ideally we achieve these requirements:

      • Retrieve flamegraph SVG generated from latest profile trace.
      • Online enable and disable of profiling activity. (async-profiler does not do instrumentation based profiling so this should not cause the code gen related perf problems of that other approach and can be safely toggled on and off while under production load.)
      • CPU profiling.
      • ALLOCATION profiling.

      Attachments

        Issue Links

          Activity

            People

              vjasani Viraj Jasani
              vjasani Viraj Jasani
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m