Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-27318

Hive CLI creates 2 Tez sessions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.3
    • 3.1.4
    • Hive, Tez
    • Bigtop 3.2.0 on Debian 10

    Description

      Running Hive CLI I noticed in the new version of Hive 3.1.3 and Tez 0.10.1 in Bigtop 3.2.0 (as opposed to Bigtop 1.5.0), there are now 2 Tez sessions created with "hive.execution.engine=tez" enabled as default:

      2023-05-04T13:46:19,189  INFO [main] SessionState: Hive Session ID = dbd5b26f-9904-44a1-b70b-9477287e0c03
      
      2023-05-04T13:46:22,275  INFO [pool-6-thread-1] SessionState: Hive Session ID = 58d15cbd-f815-4c55-b323-02559dcacd61

      When compared to previous behavior, there was only 1 session created with the main thread. Worst thing is that the session created by the pool thread is not even utilized at all, and will just linger and take up resources for the ApplicationMaster container, until it dies from timeout defined in "tez.session.am.dag.submit.timeout.secs".

      Running queries also only utilizes the session created by the main thread ie.

       

      Status: Running (Executing on YARN cluster with App id application_1682588680048_0135)
      2023-05-04T13:50:18,804  INFO [dbd5b26f-9904-44a1-b70b-9477287e0c03 main] SessionState: Status: Running (Executing on YARN cluster with App id application_1682588680048_0135)
      

      This is using identical configs in Hive for both Hive 3.1.3 and Hive 2.3.6.

      This behavior is not present in Hiveserver2 though, and could only be observed when using the Hive CLI.

      Attachments

        Issue Links

          Activity

            People

              sercan.tekin Sercan Tekin
              jeromewu Jing-Long Wu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m