Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-2115

Incorrect behavior of HMsFollower when HDFSSync feature is disabled.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.0
    • Component/s: None
    • Labels:
      None

      Description

      Current Behavior,

      Scenario-1: When HDFS sync is disabled, and sentry is started for the first time.

      • Sentry would take a full snapshot of HMS and just persists the event-id of the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID. This is wrong

      Scenario-2: When HDFS sync is disabled, and current event-id from HMS is less than last event-d processed by sentry

      • Sentry would take a full snapshot of HMS and just persists the event-id of the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID. This is wrong

      Scenario-3: When HDFS sync is disabled, and first event-id in the subsequent pull is not greater than the last event-id processed by sentry by 1.

      • Sentry would take a full snapshot of HMS and just persists the event-id of the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID. This is wrong

      Scenario-4: Initially HDFS sync was enabled and later disabled for while and then HDFS sync is enabled and sentry service is restarted to get it to effect.

      • On disabling HDFS sync, HMSFollower would update the SENTRY_HMS_NOTIFICATION_ID table but not the MSentryPathChange table.
        When HDFS sync is enabled again, HMSFollower would continue fetching new notifications and process them and update the MSentryPathChange and MAuthzPathsMapping info. This is not correct as sentry would not take a snapshot and will not have any path-mapping information about HMS objects. As a result, HDFS will ACL will not be added for the existing HMS objects. This is wrong.

      Correct Behavior:

      • Full snapshots need not be taken in all Scenario-1, Scenario-2 and Scenario-3.
      • When Sentry detects out-of-sync situations, it should reset SENTRY_HMS_NOTIFICATION_ID table and start processing the event in HMS_NOTIFICATION_LOG from beginning.
      • To handle scenario explained in Scenario-4, sentry should reset the mapping information when ever HDFS sync is disabled. That way it can learn from scratch when the feature is enabled back. There is no value is holding stale data even when we know it will have issues when the feature is enabled back.

        Attachments

        1. SENTRY-2115.004.patch
          45 kB
          Krishna Kalyan
        2. SENTRY-2115.003.patch
          46 kB
          Krishna Kalyan
        3. SENTRY-2115.002.patch
          36 kB
          Krishna Kalyan
        4. SENTRY-2115.001.patch
          16 kB
          Krishna Kalyan

          Issue Links

            Activity

              People

              • Assignee:
                kkalyan Krishna Kalyan
                Reporter:
                kkalyan Krishna Kalyan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: