Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1564

Metadata for dead Oozie servers should remain in ZK

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • trunk
    • None
    • HA
    • None

    Description

      As per the discussion in OOZIE-1561, we should have two services in the service discovery on ZK:

      1. Contains the metadata for any Oozie server that has ever connected, even ones that are currently dead
        • This would be used by log streaming so that dead servers are always identified
        • I think there is an easy way to do this with a setting in Curator, but I have to test it. If not, then we'll have to manually create/manage the ZNodes.
      2. Contains the metadata for any Oozie server that is still alive (the current behavior)
        • This would be used by the mod calculation so only alive servers are included

      Maintaining two lists will avoid the need to do any pinging or tricky heartbeating.

      We'd have to add a new admin command to cleanup the dead servers. It would look at which servers are not in the alive list and remove those.

      It would also be good to update the admin command that lists servers to specify which ones are alive and which are dead; and possibly have filtering (i.e. show only alive servers, show only dead servers, show all servers)

      Attachments

        Issue Links

          Activity

            People

              bowenzhangusa Bowen Zhang
              rkanter Robert Kanter
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: