Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11390

Describe formatted statement on materialized view should show the view definition

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 4.3.0
    • Frontend
    • None

    Description

      After IMPALA-10723, materialized views are treated as table instead of view in Impala. The DESCRIBE FORMATTED statement therefore is missing the view information.

      Results in Impala

      [localhost:21050] default> describe formatted functional_orc_def.mv1_alltypes_jointbl;
      Query: describe formatted functional_orc_def.mv1_alltypes_jointbl
      +------------------------------+------------------------------------------------------------------------------------------+----------------------+
      | name                         | type                                                                                     | comment              |
      +------------------------------+------------------------------------------------------------------------------------------+----------------------+
      | # col_name                   | data_type                                                                                | comment              |
      |                              | NULL                                                                                     | NULL                 |
      | c1                           | smallint                                                                                 | NULL                 |
      | c2                           | boolean                                                                                  | NULL                 |
      | c3                           | bigint                                                                                   | NULL                 |
      | min_bigint                   | bigint                                                                                   | NULL                 |
      | min_zip                      | int                                                                                      | NULL                 |
      |                              | NULL                                                                                     | NULL                 |
      | # Detailed Table Information | NULL                                                                                     | NULL                 |
      | Database:                    | functional_orc_def                                                                       | NULL                 |
      | OwnerType:                   | USER                                                                                     | NULL                 |
      | Owner:                       | quanlong                                                                                 | NULL                 |
      | CreateTime:                  | Sun May 22 11:16:57 CST 2022                                                             | NULL                 |
      | LastAccessTime:              | UNKNOWN                                                                                  | NULL                 |
      | Retention:                   | 0                                                                                        | NULL                 |
      | Location:                    | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL                 |
      | Table Type:                  | MATERIALIZED_VIEW                                                                        | NULL                 |
      | Table Parameters:            | NULL                                                                                     | NULL                 |
      |                              | bucketing_version                                                                        | 2                    |
      |                              | materializedview.engine                                                                  | tez                  |
      |                              | numFiles                                                                                 | 1                    |
      |                              | totalSize                                                                                | 555                  |
      |                              | transient_lastDdlTime                                                                    | 1653189458           |
      |                              | NULL                                                                                     | NULL                 |
      | # Storage Information        | NULL                                                                                     | NULL                 |
      | SerDe Library:               | org.apache.hadoop.hive.ql.io.orc.OrcSerde                                                | NULL                 |
      | InputFormat:                 | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat                                          | NULL                 |
      | OutputFormat:                | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat                                         | NULL                 |
      | Compressed:                  | No                                                                                       | NULL                 |
      | Num Buckets:                 | -1                                                                                       | NULL                 |
      | Bucket Columns:              | []                                                                                       | NULL                 |
      | Sort Columns:                | []                                                                                       | NULL                 |
      |                              | NULL                                                                                     | NULL                 |
      | # Constraints                | NULL                                                                                     | NULL                 |
      +------------------------------+------------------------------------------------------------------------------------------+----------------------+
       

      Results in Hive (the view definition is shown as Original Query):

      +-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
      |                   col_name                    |                     data_type                      |                      comment                       |
      +-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
      | c1                                            | smallint                                           |                                                    |
      | c2                                            | boolean                                            |                                                    |
      | c3                                            | bigint                                             |                                                    |
      | min_bigint                                    | bigint                                             |                                                    |
      | min_zip                                       | int                                                |                                                    |
      |                                               | NULL                                               | NULL                                               |
      | # Detailed Table Information                  | NULL                                               | NULL                                               |
      | Database:                                     | functional_orc_def                                 | NULL                                               |
      | OwnerType:                                    | USER                                               | NULL                                               |
      | Owner:                                        | quanlong                                           | NULL                                               |
      | CreateTime:                                   | Sun May 22 11:16:57 CST 2022                       | NULL                                               |
      | LastAccessTime:                               | UNKNOWN                                            | NULL                                               |
      | Retention:                                    | 0                                                  | NULL                                               |
      | Location:                                     | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL                                               |
      | Table Type:                                   | MATERIALIZED_VIEW                                  | NULL                                               |
      | Table Parameters:                             | NULL                                               | NULL                                               |
      |                                               | bucketing_version                                  | 2                                                  |
      |                                               | materializedview.engine                            | tez                                                |
      |                                               | numFiles                                           | 1                                                  |
      |                                               | totalSize                                          | 555                                                |
      |                                               | transient_lastDdlTime                              | 1653189458                                         |
      |                                               | NULL                                               | NULL                                               |
      | # Storage Information                         | NULL                                               | NULL                                               |
      | SerDe Library:                                | org.apache.hadoop.hive.ql.io.orc.OrcSerde          | NULL                                               |
      | InputFormat:                                  | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat    | NULL                                               |
      | OutputFormat:                                 | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat   | NULL                                               |
      | Compressed:                                   | No                                                 | NULL                                               |
      | Num Buckets:                                  | -1                                                 | NULL                                               |
      | Bucket Columns:                               | []                                                 | NULL                                               |
      | Sort Columns:                                 | []                                                 | NULL                                               |
      |                                               | NULL                                               | NULL                                               |
      | # Materialized View Information               | NULL                                               | NULL                                               |
      | Original Query:                               | SELECT t1.smallint_col c1, t1.bool_col c2,         | NULL                                               |
      |                                               |                                                    | t2.test_id c3, min(t1.bigint_col) min_bigint, min(t2.test_zip) min_zip |
      |                                               |                                                    | FROM functional_orc_def.alltypes t1                |
      |                                               |                                                    | JOIN functional_orc_def.jointbl t2 ON (t1.id=t2.alltypes_id) |
      |                                               |                                                    | group by t1.smallint_col, t1.bool_col, t2.test_id  |
      | Expanded Query:                               | SELECT `t1`.`smallint_col` `c1`, `t1`.`bool_col` `c2`, | NULL                                               |
      |                                               |                                                    | `t2`.`test_id` `c3`, min(`t1`.`bigint_col`) `min_bigint`, min(`t2`.`test_zip`) `min_zip` |
      |                                               |                                                    | FROM `functional_orc_def`.`alltypes` `t1`          |
      |                                               |                                                    | JOIN `functional_orc_def`.`jointbl` `t2` ON (`t1`.`id`=`t2`.`alltypes_id`) |
      |                                               |                                                    | group by `t1`.`smallint_col`, `t1`.`bool_col`, `t2`.`test_id` |
      | Rewrite Enabled:                              | Yes                                                | NULL                                               |
      | Outdated for Rewriting:                       | No                                                 | NULL                                               |
      |                                               | NULL                                               | NULL                                               |
      | # Materialized View Source table information  | NULL                                               | NULL                                               |
      | Table name                                    | I/U/D since last rebuild                           | NULL                                               |
      | hive.functional_orc_def.alltypes              | 0/0/0                                              | NULL                                               |
      | hive.functional_orc_def.jointbl               | 0/0/0                                              | NULL                                               |
      +-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+

      CC amansinha

      Attachments

        Issue Links

          Activity

            People

              pengdou1990 pengdou1990
              stigahuang Quanlong Huang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: