Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10947

SQL support for querying Iceberg metadata

    XMLWordPrintableJSON

Details

    • Epic
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Frontend

    Description

      HIVE-25457 added support for querying Iceberg table metadata to Hive.

      They support the following syntax:
      SELECT * FROM default.iceberg_table.history;

      Spark uses the same syntax: https://iceberg.apache.org/spark-queries/#history

      Other than "history", the following metadata tables are available in Iceberg:
      The following metadata tables are available in Iceberg:

      • ENTRIES,
      • FILES,
      • HISTORY,
      • SNAPSHOTS,
      • MANIFESTS,
      • PARTITIONS,
      • ALL_DATA_FILES,
      • ALL_MANIFESTS,
      • ALL_ENTRIES

      Impala currently only supports "DESCRIBE HISTORY <tbl>". The above SELECT syntax would be more convenient for the users, also it would be more flexible as users could easily define filters in WHERE clauses. And of course we would be consistent with other engines.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-12661 ASAN heap-use-after-free in IcebergMetadataScanNode

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-12706 Failing DCHECK when querying STRUCT inside a STRUCT for Iceberg metadata table

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-12853 Collection types in FROM clause for Iceberg Metadata Tables

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-13035 Querying metadata tables from non-Iceberg tables throws IllegalArgumentException

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-12845 Crash with DESCRIBE on a complex type from an Iceberg table

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Documentation - Documentation or Website IMPALA-13036 Document Iceberg metadata tables

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              daniel.becker Daniel Becker
              boroknagyz Zoltán Borók-Nagy
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: