Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-6517 Snapshot support for Ozone
  3. HDDS-7951

[Snapshot] Clean SnapDiff job and report table

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • None

    Description

      Problem: For snapshot diff report persistence, we use RocksDB. Since RocksDB doesn't expose any way to get ColumnFamilyHandle for existing table which is needed to support pagination and we decided to have a global table snap-diff-report-table to store diff report where key will be prefixed with ReportId or JobId. Over the time this table will grow and will needed to clean.

      Task: Expectation from this is to remove entries from snap-diff-report-table for a job once persistence time is over (like 7 or 30 days which is configurable) or job failed due OM crash or any other reason.

      Approach: One approach could be following.
      We have another table snap-diff-job-table which can be used to determine which job and thier report can be removed. It would be based on time instance (creation time) and the job status (FAILED). Iterator over this table and move entries from table: snap-diff-job-table to some new table (may be something delete-snap-diff-report-table). Later deleted all the reports related to entries in table delete-snap-diff-report-table.

      Attachments

        Issue Links

          Activity

            People

              hemantk Hemant Kumar
              hemantk Hemant Kumar
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: