TestNGTestClassProcessorTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Start migrating test classes to the most appropriate subproject

Story: gradle/langos#103

Item: refactor-plugins

    • -417
    • +0
    ./TestNGTestClassProcessorTest.groovy
  1. … 124 more files in changeset.
Moved validation of TestNG `configFailurePolicy` from the test processor to the test framework. This means the task will fail early when the requested policy is not supported, rather than attempting to start workers and run tests.

  1. … 4 more files in changeset.
Changed TestNGTestResultProcessorAdapter to generate start/complete events for TestNG suite execution, and removed the synthetic event generation from TestNGTestClassProcessor.

    • -35
    • +117
    ./TestNGTestClassProcessorTest.groovy
  1. … 5 more files in changeset.
Inject a `TimeProvider` into the various test processors and result handlers that need to take timestamps.

  1. … 7 more files in changeset.
GRADLE-2841 Removed unused fields

  1. … 5 more files in changeset.
Added coverage for TestNG suites processing

    • -3
    • +28
    ./TestNGTestClassProcessorTest.groovy
  1. … 1 more file in changeset.
Improved the TestNG listeners a little bit. Now any custom TestNG listeners are applied before Gradle's listeners. This way, the custom listener is more robust and can change test status. There might be some impact of this change hence mentioned in the notes.

    • -0
    • +30
    ./TestNGTestClassProcessorTest.groovy
  1. … 2 more files in changeset.
Tightened up some test expectations.

REVIEW-3641 Added some extra coverage.

    • -61
    • +31
    ./TestNGTestClassProcessorTest.groovy
Reworked the fine test selection DSL according to feedback / discussion.

  1. … 20 more files in changeset.
Some work with single test execution DSL. Renamed 'selection' -> 'filter'.

  1. … 11 more files in changeset.
Reworked the test selection dsl and implementation for simplicity and for much easier appliance of --only command line option.

  1. … 16 more files in changeset.
Single test method execution - the test / class matching is now based on simple glob than on regexp. This way it is more user friendly and also avoids some problems with regexps.

  1. … 9 more files in changeset.
Made TestNg support class name pattern and method pattern.

    • -11
    • +18
    ./TestNGTestClassProcessorTest.groovy
  1. … 4 more files in changeset.
Reworked the single test method execution so that the test framework receives a test class + test method spec. This is needed to provide decent command line support later on.

    • -3
    • +22
    ./TestNGTestClassProcessorTest.groovy
  1. … 16 more files in changeset.
Checkstyle

Reworked the class a little bit so it's cleaner, less lines and more groovy.

    • -138
    • +75
    ./TestNGTestClassProcessorTest.groovy
Added lower level integ test coverage for JUnit single test method execution

  1. … 2 more files in changeset.
Enabled single test execution for TestNG, too.

    • -1
    • +49
    ./TestNGTestClassProcessorTest.groovy
  1. … 4 more files in changeset.
REVIEW-1290 rename TestNGOptionsPojo to TestNGSpec; simplify errorhandling in TestNGTestFramework

  1. … 5 more files in changeset.
Dont use groovy in TestNGTestClassProcessor.

  1. … 4 more files in changeset.
Some renaming and cleanup around the test/temp directory used in tests.

  1. … 308 more files in changeset.
TestNG creates xml results efficiently - an initial implementation.

1. The xml results are generated by the parent process and not by the test worker process. To avoid increased memory footprint, the test logging (std out/err, log4j, etc.) output is written to files (with buffering). To avoid speed regression, the test logging is copied to the resulting xml (e.g. no deserialization, just copying bytes from input stream to output stream). This way the new xml reports take less memory than previous Gradle versions without incurring build speed regression.

2. After many profiling sessions I've decided to not serialize the test results just yet (only the test logging is serialized). The test results (e.g. set of class names, test method names with results information) does not seem to consume too much memory at this stage. The full performance test run will soon validate this statement.

3. The TestNG exection generates 2 extra files per every test class. Those 2 files contain test output and test error. For JUnit I will be able to tweak the algorithm to generate less files and build some of the reports on the fly. It's because with JUnit we receive notifications on test class start / end. Hence, the JUnit xml results generation should be even more efficient, once all stories are completed.

4. The xml is written using the sax api to make sure it is efficient. The test outputs are also streamed.

5. Writing the test output to the files uses some simple caching of the open files + buffered writer.

6. Efficient generation of the html results may be nicely blended into above approach, possibly generating some data on the fly.

  1. … 26 more files in changeset.
Spockified a test

    • -239
    • +124
    ./TestNGTestClassProcessorTest.groovy
  1. … 1 more file in changeset.
TestNG reports are now lovely (not by default, though)

1. By default the TestNG works as it was before, e.g. all 3 old reports are generated into usal spots. Those reports are generated by *TestNG*. Those reports do not contain test output and fail when forkEvery or maxParallelForks is used.

2. Only if the user configures the test.testReport = true in his buildscript the new reports are generated. If testReport is on, the old TestNG reports are *not* generated. It is somewhat a breaking change so it needs proper documentation and announcement. The new reports are generated by *Gradle* which means they are beautiful, they contain test output and work smoothly with forkEvery / maxParallelForks. The new reports consists of: new xml results (with test outputs, CI servers will be happy), new html reports (pretty, easy to read and navigate).

3. If (unlikely) some user really wants to generate both reports (old and new) it is possible, he needs to configure the default listeners via TestNGOptions.listeners

4. Setting TestNGOptions.useDefultListeners = true has no effect if the testReport is on. A mildly breaking change.

5. It is possible to prevent generation of both old and new reports via TestNGOptions.useDefultListeners = false and test.testReport = false

The new reports are not yet turned on by default because they consume tons of heap space. It's because with TestNG we don't receive events for start/end test class and hence we need to wait until the suite end to generate the xml results. This problem can be fixed, for example by making the serialization smarter. However, it's not yet implemented.

The proposed breaking changes are mild IMHO. The default behavior stays untouched. Also, the migration plan to use the new reports by default should be fairly straightforward. The changes are more-less what we've discussed earlier with Adam; also discussed with PeterN today. Please review and thanks for reading :)

    • -18
    • +12
    ./TestNGTestClassProcessorTest.groovy
  1. … 10 more files in changeset.
Moved IdGenerator and implementations from core to baseServices.

  1. … 53 more files in changeset.
Renamed subprojects/gradle-(.+) to subprojects/$1

    • -0
    • +416
    ./TestNGTestClassProcessorTest.groovy
  1. … 6178 more files in changeset.