Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-25981

Avoid checking for archived parts in analyze table

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Analyze table on large partitioned table is expensive due to unwanted checks on archived data.

       

      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:3908)
          - locked <0x00000003d4c4c070> (a org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler)
          at com.sun.proxy.$Proxy56.listPartitionsWithAuthInfo(Unknown Source)
          at org.apache.hadoop.hive.ql.metadata.Hive.getPartitions(Hive.java:3845)
          at org.apache.hadoop.hive.ql.exec.ArchiveUtils.conflictingArchiveNameOrNull(ArchiveUtils.java:299)
          at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.validate(SemanticAnalyzer.java:13579)
          at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:241)
          at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
          at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:196)
          at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:615)
          at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:561)
          at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:555)
          at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)
          at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:204)
          at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:265)
          at org.apache.hive.service.cli.operation.Operation.run(Operation.java:285)  
       

      Attachments

        Issue Links

          Activity

            People

              ayushtkn Ayush Saxena
              rajesh.balamohan Rajesh Balamohan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 40m
                  1h 40m