Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3234

Catalog should send incremental metadata changes to Impalads

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • Impala 2.5.0
    • None
    • Catalog

    Description

      Currently, every table metadata change (DDL/DML operation) causes the catalog to serialize the entire table metadata and send it to: a) the impalad node that triggered the operation and b) all the impalad nodes via a statestore update. The catalog should instead send only the portion of the table metadata that changed. Furthermore, certain operations, like DESCRIBE statements, don't require the full table metadata.

      Attachments

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-4967 Metadata distribution should be incremental

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-3127 Decouple partitions from tables

          • Major - Major loss of function.
          • Resolved
          relates to

          Task - A task that needs to be done. IMPALA-2129 Catalogd support to load only table/column metadata

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              dtsirogiannis Dimitris Tsirogiannis
              Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: