Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-16740

Mini cluster test flakiness

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Mini clusters used during HDFS unit tests are reporting test failures that do not appear to be directly related to submitted changes.  The failures are the result of either interactions between tests run in parallel, or tests which share common disk space for tests.  In all cases, the tests can be run individually serially without any errors.  Addressing this issue will simplify future submissions by eliminating the confusion introduced by these unrelated test failures.

      We can apply lessons recently from TestRollingUpgrade, which was recently patched to unblock a recent submission.  The fixes involved changing the HDFS configuration to use temporary disk space for each individual tests, and using try-with-resources to ensure that clusters were shutdown cleanly.

      Attachments

        Activity

          People

            svaughan Steve Vaughan
            svaughan Steve Vaughan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: