Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12805

Avoid problems of HIVE-27114 without configuration changes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Won't Fix
    • Impala 4.4.0
    • Not Applicable
    • Catalog
    • None

    Description

      HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads.

      One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression.

      Fix this by setting a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but this is not enough to override the default. Instead we set the string “do_not_match” which should never be used for as the name of any partition stats.

      Attachments

        Activity

          People

            asherman Andrew Sherman
            asherman Andrew Sherman
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: