Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15475

Fix broken unit tests on Windows

    XMLWordPrintableJSON

Details

    • Test
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.1.0, 2.9.1
    • None
    • None

    Description

      There are hundreds of unit tests that fail on Windows. This JIRA tracks the effort to fix them.

      The main reasons for unit test failures on Windows are:

      • Windows/Linux path formats (e.g., HDFS-10256).
      • Line separator.
      • Locked files: Windows locks files when opening them.
        • The typical trigger is not cleaning MiniDFSCluster leaves files locked when a test times out; they need to be cleaned using After.
      • Memory lock size.
      • Slow DNS resolution (e.g., HDFS-13569).
      • Locked ports (e.g., HDFS-11700)

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7104 TestHistoryViewerPrinter fails on windows.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-15308 TestConfiguration fails on Windows because of paths

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15454 TestRollingFileSystemSinkWithLocal fails on Windows

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15458 TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15494 TestRawLocalFileSystemContract fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15497 TestTrash should use proper test path to avoid failing on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15498 TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15529 ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13586 Fsync fails on directories on Windows

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-142 TestMetadataStore fails on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13268 TestWebHdfsFileContextMainOperations fails on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13296 GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13336 Test cases of TestWriteToReplica failed in windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13408 MiniDFSCluster to support being built on randomized base directory

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13509 Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13537 TestHdfsHelper does not generate jceks path properly for relative path in Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13588 Fix TestFsDatasetImpl test failures on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13618 Fix TestDataNodeFaultInjector test failures on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8324 Fix TestPrivilegedOperationExecutor.testExecutorPath on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8327 Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8344 Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8348 Incorrect and missing AfterClass in HBase-tests to fix NPE failures

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8359 Exclude containermanager.linux test classes on Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13676 TestEditLogRace fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7103 Fix TestHistoryViewerPrinter on windows due to a mismatch line separator

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7108 TestFileOutputCommitter fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7111 TestNameNodeMetrics fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13714 Fix TestNameNodePrunesMissingStorages test failures on Windows

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. YARN-8526 Fix TestNodeManagerResync.testContainerResourceIncreaseIsSynchronizedWithRMResync failure on Windows

          • Major - Major loss of function.
          • Open

          Test - A new unit, integration or system test. YARN-8527 Fix test case failures of TestResourceTrackerService on Windows

          • Major - Major loss of function.
          • Open

          Test - A new unit, integration or system test. HDFS-13624 TestHDFSFileSystemContract#testAppend times out on Windows on the first run

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Test - A new unit, integration or system test. HDFS-13630 testUpgradeCommand fails following tests in TestDFSAdminWithHA on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Test - A new unit, integration or system test. HDFS-13503 Fix TestFsck test failures on Windows

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13648 Fix TestGetConf#testGetJournalNodes on Windows due to a mismatch line separator

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13681 Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. HADOOP-15467 TestDoAsEffectiveUser#testRealUserSetup,TestDoAsEffectiveUser#testRealUserAuthorizationSuccess time out on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HADOOP-15496 TestFsShellList#testList fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HADOOP-15532 TestBasicDiskValidator fails with NoSuchFileException

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-11700 TestHDFSServerPorts#testBackupNodePorts doesn't pass on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13542 TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13548 TestResolveHdfsSymlink#testFcResolveAfs fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13550 TestDebugAdmin#testComputeMetaCommand fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13551 TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13552 TestFileAppend.testAppendCorruptedBlock,TestFileAppend.testConcurrentAppendRead time out on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13554 TestDatanodeRegistration#testForcedRegistration does not shut down cluster

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13555 TestNetworkTopology#testInvalidNetworkTopologiesNotCachedInHdfs times out on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13556 TestNestedEncryptionZones does not shut down cluster

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13557 TestDFSAdmin#testListOpenFiles fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13558 TestDatanodeHttpXFrame does not shut down cluster

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13559 TestBlockScanner does not close TestContext properly

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13560 Insufficient system resources exist to complete the requested service for some tests on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13561 TestTransferFsImage#testGetImageTimeout times out intermittently on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13562 TestPipelinesFailover times out on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13563 TestDFSAdminWithHA times out on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13567 TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13568 TestDataNodeUUID#testUUIDRegeneration times out on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13569 TestDataNodeMultipleRegistrations#testClusterIdMismatchAtStartupWithHA times out intermittently on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13570 TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13587 TestQuorumJournalManager fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13591 TestDFSShell#testSetrepLow fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13592 TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13593 TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13619 TestAuditLoggerWithCommands fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13620 Randomize the test directory path for TestHDFSFileSystemContract

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13627 TestErasureCodingExerciseAPIs fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13629 Some tests in TestDiskBalancerCommand fail on Windows due to MiniDFSCluster path conflict and improper path usage

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13631 TestDFSAdmin#testCheckNumOfBlocksInReportCommand should use a separate MiniDFSCluster path

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13632 Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13649 Randomize baseDir for MiniDFSCluster in TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. HDFS-13651 TestReencryptionHandler fails on Windows

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. MAPREDUCE-7102 Fix TestJavaSerialization for Windows due a mismatch line separator

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. MAPREDUCE-7105 Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. YARN-8370 Some Node Manager tests fail on Windows due to improper path/file separator

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Test - A new unit, integration or system test. YARN-8422 TestAMSimulator failing with NPE

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              huanbang1993 Anbang Hu
              huanbang1993 Anbang Hu
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: