Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12902

Event replication can be broken if hms_event_incremental_refresh_transactional_table=false

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Catalog
    • None
    • ghx-label-3

    Description

      when setting hms_event_incremental_refresh_transactional_table=false metadata.test_event_processing.TestEventProcessing.test_event_based_replication fails at the following assert:

      https://github.com/apache/impala/blob/6c0c26146d956ad771cee27283c1371b9c23adce/tests/metadata/test_event_processing_base.py#L234

       

      Based on the logs catalogd only sees alter_database and transaction events in this case, so if the transaction events (COMMIT_TXN) are ignore, then it doesn't detect the change in the table.

      This seems strange as the commit that added the test is older than the one that added hms_event_incremental_refresh_transactional_table

      https://github.com/apache/impala/commit/e53d649f8a88f42a70237fe7c2663baa126fed1a

      vs

      https://github.com/apache/impala/commit/097b10104f23e0927d5b21b43a79f6cc10425f59

       

      So it is not clear to me how could the test pass originally. One possibility is that different events were generated in HMS at that time. 

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              csringhofer Csaba Ringhofer
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: