Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-25656

Get materialized view state based on number of affected rows of transactions

    XMLWordPrintableJSON

Details

    Description

      To enable the faster incremental rebuild of materialized views presence of update/delete operations on the source tables of the view since the last rebuild must be checked. Based on the outcome different plan is generated for scenarios in presence of update/delete and insert only operations.

      Currently this is done by querying the COMPLETED_TXN_COMPONENTS table however the records from this table is cleaned when MV source tables are compacted. This reduces the chances of incremental MV rebuild.

      The goal of this patch is to find an alternative way to store and retrieve this information.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-25878 Unable to compile cpp metastore thrift client

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-26375 Invalid materialized view after rebuild if source table was compacted

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-25745 Print transactional stats of materialized view source tables

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-25747 Make a cost base decision when rebuilding materialized views

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HIVE-25744 Support backward compatibility of thrift struct CreationMetadata

          • Major - Major loss of function.
          • Closed
          is required by

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-11390 Describe formatted statement on materialized view should show the view definition

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #2756

          Activity

            People

              kkasa Krisztian Kasa
              kkasa Krisztian Kasa
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m