Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
When I use Junit 5 to run the tests in parallel (not using Surefire Forking), the console logs are confusing when the test failed.
As per Junit 5 documentation, when you use Junit 5 for parallelism, the tests will run using different Forking threads in the same process. And, when there is a test failure, Surefire is logging respective test failure logs. Instead, it is just picking random logs from different threads.
Would it be possible to get the right logs for failed tests?
Junit 5 Configuration from Surefire Plugin:
Tests run:
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ api-test ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.test.abc.ABCTests
[INFO] Running com.test.abc.XYZTests
[INFO] Running com.test.abc.ATests
[INFO] Running com.test.abc.GetLogsTests
[INFO] Running com.test.abc.FoldersTests
[INFO] Running com.test.abc.NameTests
Example Logs from Console:
NFO |1026-103227268|ForkJoinPool-1-worker-3|abc.ABCTests||||||About to execute test [delegatedUserAccount] - GUID [random-guid] and data [[1] com.test.abc.ABCTestData@37589144] INFO |1026-103227268|ForkJoinPool-1-worker-13|abc.XYZTests||||||About to execute test [getTestId] - GUID [random-guid] and data [[1] com.test.abc.data.XYZTestData@4c506080] INFO |1026-103227268|ForkJoinPool-1-worker-5|abc.ATests||||||About to execute test [getFolderTest] - GUID [0e1141b0-3445-36f7-879a-2f12f3de87a7] and data [[1] com.test.abc.data.FoldersTestData@484d6599] INFO |1026-103227268|ForkJoinPool-1-worker-7|abc.GetLogsTests||||||About to execute test [getIds] - GUID [random-guid] and data [[1] com.test.abc.data.GetLogsTestData@4cf26b0c]
When there is any test failure from the execution, we see logs from all different threads like below...
INFO |1026-103227268|ForkJoinPool-1-worker-3|abc.ABCTests||||||About to execute test [delegatedUserAccount] - GUID [random-guid] and data [[1] com.test.abc.ABCTestData@37589144] INFO |1026-103227268|ForkJoinPool-1-worker-13|abc.XYZTests||||||About to execute test [getTestId] - GUID [random-guid] and data [[1] com.test.abc.data.XYZTestData@4c506080] INFO |1026-103227268|ForkJoinPool-1-worker-5|abc.ATests||||||About to execute test [getFolderTest] - GUID [0e1141b0-3445-36f7-879a-2f12f3de87a7] and data [[1] com.test.abc.data.FoldersTestData@484d6599] INFO |1026-103227268|ForkJoinPool-1-worker-7|abc.GetLogsTests||||||About to execute test [getIds] - GUID [random-guid] and data [[1] com.test.abc.data.GetLogsTestData@4cf26b0c]
Expected Behavior
It would be good to have console logs at the Test case level rather than picking up from random threads though we run tests using Junit 5 or Surefire Forks (JVMs).
Steps To Reproduce
Use Junit 5 Configuration from Maven Surefire Plugin v3.0.0-M5 with JDK11.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.plugin.version}</version>
<configuration>
<trimStackTrace>false</trimStackTrace>
<includes>
<include>**/*Test.java</include>
<include>**/*Tests.java</include>
</includes>
<properties>
<!--includeTags & excludeTags don't work with Surefire v3.0.0-M5.
Same thing now works with groups (Groups are nothing but Tags)-> means all Included Tags,
excludedGroups -> means Excluded Tags. While running the tests using maven command
with the help of Tags,either we should use -DTags/-Dgroups-->
<groups>${Tags}</groups>
<excludedGroups>Skip, WIP</excludedGroups>
<!-- Parallel execution using Junit 5 -->
<configurationParameters>
junit.jupiter.conditions.deactivate = *
junit.jupiter.extensions.autodetection.enabled = true
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.config.strategy=fixed
junit.jupiter.execution.parallel.config.fixed.parallelism=6
junit.jupiter.testinstance.lifecycle.default = per_class
</configurationParameters>
</properties>
<systemPropertyVariables>
<!-- https://bugs.openjdk.java.net/browse/JDK-8212755 -->
<java.locale.providers>COMPAT</java.locale.providers>
</systemPropertyVariables>
</configuration>
</plugin>
Run the tests using the maven command to run the tests in parallel using Tags...
mvn -Dgroups=TestingTag