Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-26559

Skip unnecessary get all partition operations when where condition with 1=0 in CBO.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • All Versions
    • None
    • CBO
    • None

    Description

      In some cases, queries may get executed with where condition mentioning to "1=0" to get schema. E.g

      SELECT
        *
      FROM
        table_with_millions_of_partitions
      WHERE
        1=0
      

      In actual production, it likes:

      SELECT
        *
      FROM
        table_with_millions_of_partitions
      WHERE
        partition_col1 = value1
        and partition_col2 = value2
        and if(some conditions, true, false)
      

       

      When the cbo optimizer optimizes the execution plan of this query, the cbo optimizer will get all the partitions of table_with_millions_of_partitions. This seems useless and causes hiveserver to fail when the number of partitions is very high.

       

      Use this patch to skip unnecessary get all partition operation when pruneNode is always false.

      Attachments

        1. HIVE-26559.patch
          2 kB
          shuaiqi.guo

        Issue Links

          Activity

            People

              shuaiqi.guo shuaiqi.guo
              shuaiqi.guo shuaiqi.guo
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 1h
                  1h
                  Remaining:
                  Remaining Estimate - 1h
                  1h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified