JavaConfigurationPerformanceTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Reduce number of no-daemon tests

Make using the daemon the default in performance tests

and reduce the number of no-daemon tests so their maintenance

becomes easier. The daemon is the default mode for Gradle and

we are generally ready to trade better daemon performance for

slightly worse no-daemon performance.

    • -47
    • +0
    ./JavaConfigurationPerformanceTest.groovy
  1. … 30 more files in changeset.
Bump up memory limits for some performance tests

    • -4
    • +4
    ./JavaConfigurationPerformanceTest.groovy
  1. … 1 more file in changeset.
Slightly increase memory for bigOldJava, as 3.2.1 also struggles with 512m

    • -1
    • +1
    ./JavaConfigurationPerformanceTest.groovy
  1. … 6 more files in changeset.
Reset performance test memory for large projects

    • -1
    • +1
    ./JavaConfigurationPerformanceTest.groovy
  1. … 6 more files in changeset.
Further stabilize performance tests

    • -2
    • +2
    ./JavaConfigurationPerformanceTest.groovy
  1. … 12 more files in changeset.
Reset performance test baselines

Performance tests are automatically run against the last release,

there is no need to specify that explicitly. Only if there was an

acceptable regression should we explicitly specify a nightly.

We'll probably have to add a few of those nightly versions back in,

but many of them were just added in an attempt to stabilize tests,

not because of actual regressions.

Regressions that we accepted but might want to get back to will be

handled by the historical performance tests.

    • -6
    • +5
    ./JavaConfigurationPerformanceTest.groovy
  1. … 24 more files in changeset.
Reduce memory for performance scenarios

The scenarios should only have an amount of memory that is

"reasonable" for what they are doing. This serves two purposes.

It allows us to detect large memory regressions, as a reasonable

upper limit will lead to lots of GC time if that limit is breached.

It also makes test results more predictable, as too much memory means

that many test runs will not need garbage collection at all while other

test runs will have large GC cycles.

    • -5
    • +6
    ./JavaConfigurationPerformanceTest.groovy
  1. … 20 more files in changeset.
Rebase tests again to make it easier to investigate differences

    • -4
    • +4
    ./JavaConfigurationPerformanceTest.groovy
  1. … 9 more files in changeset.
Rebase flaky performance tests

    • -6
    • +3
    ./JavaConfigurationPerformanceTest.groovy
  1. … 7 more files in changeset.
Re-enable flaky performance tests

    • -9
    • +7
    ./JavaConfigurationPerformanceTest.groovy
  1. … 4 more files in changeset.
Rebase some flaky performance tests

    • -1
    • +1
    ./JavaConfigurationPerformanceTest.groovy
  1. … 3 more files in changeset.
Pass `flakiness` flag to the test runner

    • -1
    • +1
    ./JavaConfigurationPerformanceTest.groovy
  1. … 5 more files in changeset.
Mark more performance tests as flaky

    • -6
    • +8
    ./JavaConfigurationPerformanceTest.groovy
  1. … 3 more files in changeset.
Reset the baseline for one more test

The previous batch fixed 15 out of 23 failing performance tests. The

others may be related to a change in how dependencies are resolved.

    • -4
    • +7
    ./JavaConfigurationPerformanceTest.groovy
Update performance baselines

There have been sufficient performance gains which should mean that

master is faster than 3.1 which is the current latest release of

gradle. We should no longer need to be comparing to the nightly build

from before 3.1 was released.

    • -4
    • +3
    ./JavaConfigurationPerformanceTest.groovy
  1. … 11 more files in changeset.
Rebaseline for configuraiton performance tests

    • -3
    • +3
    ./JavaConfigurationPerformanceTest.groovy
Reset the baseline for more tests

- There are only a few more tests which are not currently passing

regularly.

    • -2
    • +3
    ./JavaConfigurationPerformanceTest.groovy
  1. … 5 more files in changeset.
Reintroduce gradle/gradle@784d747

- There was previously a concern that this would break the

BuildScansPerformanceTest, but that test doesn't actually use the

BaselineVersion class at all.

    • -8
    • +5
    ./JavaConfigurationPerformanceTest.groovy
  1. … 31 more files in changeset.
Revert "Make strict performance testing default."

This reverts commit 784d7476556e63c8d7f15919ae9875c3b7181aa3.

    • -5
    • +8
    ./JavaConfigurationPerformanceTest.groovy
  1. … 31 more files in changeset.
Make strict performance testing default.

- This makes it impossible to set maximum regression limits manually and

defaults to using a statistically derived allowable amount of

variance.

    • -8
    • +5
    ./JavaConfigurationPerformanceTest.groovy
  1. … 31 more files in changeset.
Remove Gradle 1.1 from baseline for JavaConfigurationPerformanceTest

- the biggest difference seems to be caused by the changes in how

measurements are now implemented. The total build time contains the

overhead of the measurement solution and it's higher on newer

versions than on Gradle 1.1

- this is a non-daemon test, there is a separate test that tests

with daemon enabled. That test doesn't contain 1.1 in baselines

    • -3
    • +3
    ./JavaConfigurationPerformanceTest.groovy
Split integration and performance tests

So far, performance tests were just 'integration tests in the performance project'.

This required some special handling to make sure that:

- performance tests are not run during integration test invocations

- integration tests are not run during performance test invocations

It also lead to performance tests being monolithic instead of close to the code they are testing.

This change makes performance tests a first class citizen in every project.

The common infrastructure for testing a full distribution (including daemon cleanup etc.) is now

separate from the integration tests and can be reused by other test types.

The performance tests live in their own source set called 'performanceTest'. They can define their own

dependencies and can be used independently from integration tests.

    • -0
    • +50
    ./JavaConfigurationPerformanceTest.groovy
  1. … 111 more files in changeset.