Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12771

Impala catalogd events-skipped may mark the wrong number

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Catalog
    • None
    • ghx-label-3

    Description

      See the description of event-skipped metric

       // total number of events which are skipped because of the flag setting or
        // in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were ignored
        // because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in the catalogd.
      

      As for CREATE and DROP event on Database/Table/Partition (Also AddPartition is inclued) when we found that the table/database when the database or table is not found in the cache then we will skip the event process and make the event-skipped metric +1.
      But I found that there is some question here for alter table and Reload event:

      • For Reload event that is not describe in the description of events-skipped, but the value is +1 when is oldevent;
      • Besides if the table is in blacklist the metric will also +1
        In summary, I think this description is inconsistent with the actual implementation.
        So can we also mark the events-skipped metric for alter partition events and modify the
        description to be all the events skipped

      Attachments

        Activity

          People

            maxwellguo Maxwell Guo
            maxwellguo Maxwell Guo
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: