Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-19092 ABFS phase 4: post Hadoop 3.4.0 features
  3. HADOOP-19093

Improve rate limiting through ABFS in Manifest Committer

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.4.0
    • None
    • fs/azure, test

    Description

      I need a load test to verify that the rename resilience of the manifest committer actually works as intended

      • test suite with name ILoadTest* prefix (as with s3)
      • parallel test running with many threads doing many renames
      • verify that rename recovery should be detected
      • and that all renames MUST NOT fail.

      maybe also: metrics for this in fs and doc update.
      Possibly; LogExactlyOnce to warn of load issues

      Attachments

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: