Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12884

Add GC for the inflight event list

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Catalog
    • None

    Description

      The inflight event list for catalog versions could have dangling items, e.g. in some catalog operations we don't trigger any HMS APIs but still add the version into the list.

      To be more robust, we should add GC to InFlightEvents:

      • For idsForInflightDmlEvents_, if the event id is smaller than the lastSycnedEventId of the event-processor, it won't be matched anymore.
      • For versionsForInflightEvents_, it's a bit complex. We don't know when the catalog version is stale. Maybe we should refactor the self-event detection to only check the catalog service id and ignore any event that have older catalog version than the current table. The possibility of two Impala clusters having the same catalog service id is pretty small.

      Attachments

        Activity

          People

            Unassigned Unassigned
            stigahuang Quanlong Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: