Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-17067

Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • hdfs
    • Reviewed

    Description

      InĀ HDFS-17030, we introduced an ExecutorService, to submit getHAServiceState() requests. We constructed the ExecutorService directly from a basic ThreadPoolExecutor, without setting allowCoreThreadTimeOut to true. Then, the core thread will be kept up and running even when the main thread exits. To fix it, one could set allowCoreThreadTimeOut to true. However, in this PR, we decide to directly use an existing executorService implementation (BlockingThreadPoolExecutorService) in hadoop instead. It takes care of setting allowCoreThreadTimeOut and also allows setting the prefix for thread names.

        private final ExecutorService nnProbingThreadPool =
            new ThreadPoolExecutor(1, 4, 1L, TimeUnit.MINUTES,
                new ArrayBlockingQueue<Runnable>(1024));
      

      A second minor issue is we did not shutdown the executorService in close(). It is a minor issue as close() will only be called when the garbage collector starts to reclaim an ObserverReadProxyProvider object, not when there is no reference to the ObserverReadProxyProvider object. The time between when an ObserverReadProxyProvider becomes dereferenced and when the garage collector actually starts to reclaim that object is out of control/under-defined (unless the program is shutdown with an explicit System.exit(1)).

      Attachments

        Issue Links

          Activity

            People

              xinglin Xing Lin
              xinglin Xing Lin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: