JavaConfigurationDaemonPerformanceTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Clean up performance tests

- remove unused Java software model tests

- remove unused native scenario tests

- move tests into appropriate packages

- remove unused test categories

- give tests more descriptive names

- remove unused test templates

    • -67
    • +0
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 103 more files in changeset.
Reduce number of no-daemon tests

Make using the daemon the default in performance tests

and reduce the number of no-daemon tests so their maintenance

becomes easier. The daemon is the default mode for Gradle and

we are generally ready to trade better daemon performance for

slightly worse no-daemon performance.

    • -4
    • +0
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 30 more files in changeset.
Slightly increase memory for bigOldJava, as 3.2.1 also struggles with 512m

    • -1
    • +1
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 6 more files in changeset.
Reset performance test memory for large projects

    • -1
    • +1
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 6 more files in changeset.
Further stabilize performance tests

    • -2
    • +2
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 12 more files in changeset.
Reset performance test baselines

Performance tests are automatically run against the last release,

there is no need to specify that explicitly. Only if there was an

acceptable regression should we explicitly specify a nightly.

We'll probably have to add a few of those nightly versions back in,

but many of them were just added in an attempt to stabilize tests,

not because of actual regressions.

Regressions that we accepted but might want to get back to will be

handled by the historical performance tests.

    • -5
    • +4
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 24 more files in changeset.
Reduce memory for performance scenarios

The scenarios should only have an amount of memory that is

"reasonable" for what they are doing. This serves two purposes.

It allows us to detect large memory regressions, as a reasonable

upper limit will lead to lots of GC time if that limit is breached.

It also makes test results more predictable, as too much memory means

that many test runs will not need garbage collection at all while other

test runs will have large GC cycles.

    • -5
    • +5
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 20 more files in changeset.
Rebase tests again to make it easier to investigate differences

    • -3
    • +3
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 9 more files in changeset.
Rebase flaky performance tests

    • -6
    • +3
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 7 more files in changeset.
Reset baselines for failing performance tests

There was a small acceptable regression introduced as part of the

implementation of gradle/task-output-cache#25 which has caused many

performance tests to begin failing.

Rather than investigating exactly which of these tests are failing due

to that change, we are just setting the baseline for all failing tests

to the first nightly build which included the change to see which

performance tests may still be failing for other reasons.

    • -8
    • +3
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 12 more files in changeset.
Reference a Jira issue as a reason for re-baselining

+review REVIEW-6090

    • -1
    • +1
    ./JavaConfigurationDaemonPerformanceTest.groovy
Added comment clarifying re-baselining

+review REVIEW-6090

    • -0
    • +7
    ./JavaConfigurationDaemonPerformanceTest.groovy
Re-baseline configure Java build ... (daemon)

This started failing since we are now tracking extra data

for up to date checks for copy tasks

+review REVIEW-6090

    • -3
    • +4
    ./JavaConfigurationDaemonPerformanceTest.groovy
Update performance baselines

There have been sufficient performance gains which should mean that

master is faster than 3.1 which is the current latest release of

gradle. We should no longer need to be comparing to the nightly build

from before 3.1 was released.

    • -2
    • +1
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 11 more files in changeset.
Reintroduce gradle/gradle@784d747

- There was previously a concern that this would break the

BuildScansPerformanceTest, but that test doesn't actually use the

BaselineVersion class at all.

    • -16
    • +5
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 31 more files in changeset.
Revert "Make strict performance testing default."

This reverts commit 784d7476556e63c8d7f15919ae9875c3b7181aa3.

    • -5
    • +16
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 31 more files in changeset.
Make strict performance testing default.

- This makes it impossible to set maximum regression limits manually and

defaults to using a statistically derived allowable amount of

variance.

    • -16
    • +5
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 31 more files in changeset.
Reset baselines to latest nightly - The margins by which these performance tests are failing are too small to justify spending engineering time investigating. We are resetting the baselines to eliminate the very minor performance drift to make the tests pass again. This will free up the engineers to focus on more strategic performance gains. - As the comments in the change suggest, we will revert to more reasonable baselines once we have achieved sufficient performance gains which will make Gradle faster than these older versions of itself.

    • -1
    • +2
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 5 more files in changeset.
Don't add MaxPermSize param on Java 8+ for running perf tests

    • -2
    • +2
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 19 more files in changeset.
Move Java SW model performance tests to experiments category

They are our longest-running tests and the features they test are not

used in production yet, so don't need to be regression-tested on every

commit.

    • -0
    • +2
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 6 more files in changeset.
Split integration and performance tests

So far, performance tests were just 'integration tests in the performance project'.

This required some special handling to make sure that:

- performance tests are not run during integration test invocations

- integration tests are not run during performance test invocations

It also lead to performance tests being monolithic instead of close to the code they are testing.

This change makes performance tests a first class citizen in every project.

The common infrastructure for testing a full distribution (including daemon cleanup etc.) is now

separate from the integration tests and can be reused by other test types.

The performance tests live in their own source set called 'performanceTest'. They can define their own

dependencies and can be used independently from integration tests.

    • -0
    • +81
    ./JavaConfigurationDaemonPerformanceTest.groovy
  1. … 111 more files in changeset.