Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
3.0.0-M4
-
None
-
None
Description
Junit5 allows dynamic creation of tests via {{ @TestTemplate }} and TestTemplateInvocationContextProvider.
If the TestTemplateInvocationContextProvider fails with an exception itself or Junit itself rejects it then surefire still marks the test as successful.
Both Intellij IDEA and the junit 5 console launcher mark the test method as failed.
Examples:
package surefirebug; import java.util.stream.Stream; import org.junit.jupiter.api.extension.ExtensionContext; import org.junit.jupiter.api.extension.TestTemplateInvocationContext; import org.junit.jupiter.api.extension.TestTemplateInvocationContextProvider; public class FailingTestTemplateInvocationContextProvider implements TestTemplateInvocationContextProvider { @Override public boolean supportsTestTemplate(ExtensionContext context) { return true; } @Override public Stream<TestTemplateInvocationContext> provideTestTemplateInvocationContexts( ExtensionContext context) { //throw new IllegalStateException(""); // throw a custom exception return Stream.of(); // this will be rejected by Junit itself with a PreconditionViolationException } }
package surefirebug; import org.junit.jupiter.api.TestTemplate; import org.junit.jupiter.api.extension.ExtendWith; @ExtendWith(FailingTestTemplateInvocationContextProvider.class) class DemonstrationTest { @TestTemplate void testOne() { System.out.println("testOne"); } }
Display in surefire:
... [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @ amps-utils --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running surefirebug.DemonstrationTest [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 s - in surefirebug.DemonstrationTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------
Display in the console runner:
╷
├─ JUnit Jupiter ✔
│ └─ DemonstrationTest ✔
│ └─ testOne() ✘ No supporting TestTemplateInvocationContextProvider provided an invocation context
└─ JUnit Vintage ✔
Failures (1):
JUnit Jupiter:DemonstrationTest:testOne()
MethodSource [className = 'surefirebug.DemonstrationTest', methodName = 'testOne', methodParameterTypes = '']
=> org.junit.platform.commons.PreconditionViolationException: No supporting TestTemplateInvocationContextProvider provided an invocation context
org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:296)
org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.validateWasAtLeastInvokedOnce(TestTemplateTestDescriptor.java:142)
org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:108)
org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:41)
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
[...]
Test run finished after 40 ms
[ 4 containers found ]
[ 0 containers skipped ]
[ 4 containers started ]
[ 0 containers aborted ]
[ 3 containers successful ]
[ 1 containers failed ]
[ 0 tests found ]
[ 0 tests skipped ]
[ 0 tests started ]
[ 0 tests aborted ]
[ 0 tests successful ]
[ 0 tests failed ]
I debugged a bit and this is how the error is communicated to surefire by junit:
RunListenerAdapter.executionFinished() receives the following data:
testIdentifier.getSource() instanceof MethodSource testExecutionResult.getStatus() == failed testExecutionResult.getThrowable() instanceof PreconditionViolationException
Attachments
Issue Links
- duplicates
-
-
- Closed
-
- relates to
-
-
- Open
-
-
SUREFIRE-1724 Support JUnit Platform DiscoverySelectors
-
- Open
-
- links to
