HtmlTestExecutionResult.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Detect special multicause exceptions

This commit fixes an issue where test reports and build scans didn't capture

all the causes of an exception, in case the "multi-cause" exception isn't

a type known of the daemon. By that, we mean a type which is not available

at runtime in the daemon process, but is used in the worker process.

Typically, the `org.opentest4j.MultipleFailuresError` type used by

JUnit is such an example: the type is used in a test process, but

not loaded in Gradle. In such cases, Gradle builds a placeholder

exception type which collects the root cause. But in case the

exception is actually collecting multiple causes like in this test

class, Gradle lost some information in the process.

Therefore, this commit introduces a heuristic to figure out if an

exception is collecting multiple causes instead of a single one.

The heuristic is kept very simple: one has to find a method which

name is either `getCauses` (like for Gradle's own default multi

cause exception) or `getFailures` (like in opentest4j), and it

has to return a collection of failures.

This will not fix all problems but it should capture a good amount

of them.

Fixes #9487

  1. … 3 more files in changeset.
Detect special multicause exceptions

This commit fixes an issue where test reports and build scans didn't capture

all the causes of an exception, in case the "multi-cause" exception isn't

a type known of the daemon. By that, we mean a type which is not available

at runtime in the daemon process, but is used in the worker process.

Typically, the `org.opentest4j.MultipleFailuresError` type used by

JUnit is such an example: the type is used in a test process, but

not loaded in Gradle. In such cases, Gradle builds a placeholder

exception type which collects the root cause. But in case the

exception is actually collecting multiple causes like in this test

class, Gradle lost some information in the process.

Therefore, this commit introduces a heuristic to figure out if an

exception is collecting multiple causes instead of a single one.

The heuristic is kept very simple: one has to find a method which

name is either `getCauses` (like for Gradle's own default multi

cause exception) or `getFailures` (like in opentest4j), and it

has to return a collection of failures.

This will not fix all problems but it should capture a good amount

of them.

Fixes #9487

  1. … 3 more files in changeset.
Add support for executing test classes in parallel via JUnit Platform

Fixes #6453.

  1. … 12 more files in changeset.
Update jsoup to 1.11.3

jsoup no longer converts Windows line separators to Unix ones. Thus, a

few test reporting test fixtures had to be adjusted.

  1. … 5 more files in changeset.
Make xctest fatal errors only fail the test and change execution failure test name

  1. … 7 more files in changeset.
Ensure that all children TestState are completed() (#4752)

* Ensure that all children TestState are completed()

With fail fast for tests enabled, some of the `TestState`s tracked

by the `StateTrackingTestResultProcessor` may never receive the

`completed()` call. When this happens, the parent/ancester

`TestState` will not include the test count and failure information.

When combined with a `--tests` filter, this can cause an erroneous

failure message that no tests matched the filter.

This change ensures that all of the children/descendents of a `testId`

(`TestState`) are `completed` before the given `testId` is processed.

Addresses: #4718

* Use ArrayList<> instead of LinkedList<>

  1. … 7 more files in changeset.
Add displayName and classDisplayName to TestDescriptor (#4425)

Add displayName and classDisplayName to TestDescriptor

This fixes https://github.com/gradle/gradle/issues/4424 and https://github.com/gradle/gradle/issues/4423

JUnit 5 introduces @DisplayName and dynamic tests, which allows users to

customize test case and test class' name. This should be taken into

account. This PR introduces `displayName` and `classDisplayName` which are

used for display. When rendering HTML reports, these two fields will be used.

    • -36
    • +85
    ./HtmlTestExecutionResult.groovy
  1. … 33 more files in changeset.
Stop test execution after first failure (#4190)

* Rough pass at stopNow() on test failure

* Updated worker TestClassProcessor.stopNow() to throw UnsupportedOperationException

* Updated MaxNParallelTestClassProcessor to keep "raw" processors for stopNow().

Updated other daemon-side TestClassProcessors to keep track if stopNow() is called.

* Added AbstractTestTask.getFailFast()

* Added some unit tests for to TestClassProcessors

* Added unit tests for stopNow()

* Rough pass at JUnitFailFastIntegrationTest

* Refactor fail fast JVM integration test.

Moved common logic/tests from JUnitFailFastIntegrationTest into AbstractJvmFailFastIntegrationSpec.

Added TestNGFailFastIntegrationTest (which extends TestNGFailFastIntegrationTest)

* Working on forkEvery fail fast test

* Converted FailFastTestListener to FailFastTestListenerInternal

* Added BlockingHttpServer.expectMaybeAndBlock()

* TestNG parallel & fail-fast integration test

* Remove unused import

* Remove unused import

* Marked test.failFast as @Input

* Added `failFast` to java_plugin in userguide

* Javadoc & formatting changes

* Updated userguide docs based on review comments

* Moved `failFast` configuration from `AbstractTestTask` to `Test` to avoid `XCTest`

* Updates from review comments

* More updates from review comments

* Reduced Mock() usage in FailFastTestListenerInternalTest

* Backed out changes to XCTestExecuter for fail fast behavior

* Fixed typo in javadoc

* Remove --no-fail-fast `@Option` from `Test`

* Reduce mocking expectations in ForkingTestClassProcessorTest

* Moved @Internal from Test.getFailFast() to AbstractTestTask

* Formatting updates

* Updated ForkingTestClassProcessor to track state more precisely to avoid stop() & stopNow() problem

* Fixed tests (again)

* Better handling of mutual exclusion between ForkingTestClassProcessor stop() and stopNow() sections

* Improved resiliency to indeterminate generated test class execution in fail fast tests

* Optimized imports

* Updated DefaultWorkerProcess.cleanup() to stop the Stoppables before aborting the execHandle

* Changed CyclicBarrierAnyOfRequestHandler.expected back to `private`

* ForkingTestClassProcessor.stoppedNow does not need to be `volatile` as it is now protected by a lock

* Added JUnitPlatformTestClassProcessor.stopNow()

* Removed mention of --no-fail-fast from user guide

* Added info on --fail-fast to release notes

* Fixed use of `testOmitted` in AbstractJvmFailFastIntegrationSpec

* Make MaxNParallelTestClassProcessor drop any processTestClass() invocations after stopNow()

* Protected critical region in ForkingTestClassProcessor.processTestClass() causing race condition with stopNow() in tests

* Debugging cleanup

  1. … 41 more files in changeset.
Minor changes for review

  1. … 5 more files in changeset.
Fix failed test after using "Display Name" in Junit 5

  1. … 10 more files in changeset.
Simplify a test where clause

  1. … 1 more file in changeset.
Add more test coverage for XcTest and testing frameworks

- When a test class is removed, make sure the intermediate report files are also removed.

- When no tests are found, make sure the test results have no classes listed.

- When tests are executed, make sure that only those tests are included in the report.

- When a test fails, make sure the failure is included in the report.

  1. … 11 more files in changeset.
Fix some tests that rely on a test executor with a specific ID

  1. … 14 more files in changeset.
include task name in test report/result folder when using java plugin

  1. … 11 more files in changeset.
Fail test execution when non-default config failure policy is used with a TestNG version that doesn't support it

+review REVIEW-5274

  1. … 8 more files in changeset.
Changed some int tests to use DefaultTestExecutionResult rather than JUnitTestExecutionResult, to use whatever the default mechanism for verifying test results happens to be.

  1. … 24 more files in changeset.
Changed the HtmlTestExecutionResult assert methods so that they actually assert stuff, and some other minor fixes in the various TestExecutionResult implementations.

    • -23
    • +25
    ./HtmlTestExecutionResult.groovy
  1. … 3 more files in changeset.
Changed the profile and test reports to generate only the index page into the destination directory, and the other stuff ends up in sub-directories.

  1. … 18 more files in changeset.
Moved some classes out of org.gradle.util.

  1. … 63 more files in changeset.
REVIEW-3092: moved toSafeFileName method to baseServices - avoids duplicate code in test fixtures - REVIEW-3090 encode # as well - REVIEW-3092 add testcoverage for toSafeFileName

  1. … 7 more files in changeset.
Add a bit more coverage to the cucumberjvm test and fix xml test result fixture.

  1. … 2 more files in changeset.
fix html report generation for test class names that contains spaces, slashes and other non supported chars in filenames

  1. … 5 more files in changeset.
Enabled generating the JUnit XML file with output per test case.

This is not complete. The actual generation of the XML based on the TestResultsProvider is unit tested and there is functional level coverage when running TestNG tests.

Coverage for JUnit is needed, as well as coverage at the intermediary parts of the results/report generation.

There are also some types to tidy up.

  1. … 17 more files in changeset.
REVIEW-1962: Use JUnit 4.5 in AssumptionIntegrationTest; fix HtmlTestExecutionResult

  1. … 1 more file in changeset.
REVIEW-1874 some repackaging/reorganizing of the JUnit related integrationtests; add covererage for JUnit samples in docs.

  1. … 83 more files in changeset.
Fix CI build - fix testclass lookup in HtmlTestExecutionResult - split test scoped ignored classes to extra package to support JUnit 4.0 in JUnitCrossVersionIntegTest

  1. … 6 more files in changeset.
tweak HtmlTestExecutionResult to provide more readable assertion error.

fix failing ci

REVIEW-1310:

- wrap HtmlTestExecutionResult and JUnitXmlTestExecutionResult in DefaultTestExecutionResult

- fix HtmlTestExecutionResult

- always add testclass to report

  1. … 3 more files in changeset.
REVIEW-1292: replace JSoup.parse(File, "UTF-8") by JSoup.parse(File, null)

  1. … 6 more files in changeset.