Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18477 Über-jira: S3A Hadoop 3.3.9 features
  3. HADOOP-18486

hadoop-aws tests to take a configurable subdir in the test bucket

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.5
    • None
    • fs/s3, test
    • None

    Description

      For parallel jenkins runs (now possible with s3guard cut) we need to be able to run the failsafe test suites either in separate buckets brackets (which doesn't scale) or in subdirectories of a single bucket. That needs a way to pass in a base directory for the suites and to disable cleanup of the entire bucket afterwards. Option pick up is straightforward; we will do it the way we have done with the others –an auth-keys option which can be overridden on the command line.

      S3ATestUtils.createTestPath(Path defVal) is already used to create test paths; it can be extended to pick up a unique process id as well as thread id system property, and use that for the path it constructs.

      another system property/config option is needed to disable root dir tests (ITestMarkerToolRootOperations, ITestS3AContractRootDir). a config/property picked up by the S3AContract returning false when probed for "test.root-tests-enabled" should do most of that.

      Attachments

        Activity

          People

            Unassigned Unassigned
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: