AbstractCrossVersionPerformanceTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Wire integration test build context instance

- enables using performance test specific build context when an instance

is properly wired

    • -2
    • +5
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 48 more files in changeset.
Add ForkingUnderDevelopmentGradleDistribution and use it for perf tests

- reduces differences between master vs. snapshot versions

in performance test execution

    • -2
    • +2
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 6 more files in changeset.
Adds an Android performance test

This commit introduces a new performance test for Android builds. Unlike traditional performance tests,

the "templates" of Android builds are real, external, projects checked out from Git. They are tweaked

to allow exeuction on the latest (development) version of Gradle, and add the traditional measurements

(CPU, heap).

Adding an Android performance test requires extending the `AbstractAndroidPerformanceTest` class, which

itself is a cross-version performance test.

This first step adds a single "medium" Android build as an experiment. It's worth noting that the

performance test will only execute if:

- the Android SDK is installed and path is set through the `ANDROID_HOME` environment variables

- the `$ANDROID_HOME/licenses` directory contains the accepted license files

    • -1
    • +1
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 6 more files in changeset.
Reintroduce gradle/gradle@784d747

- There was previously a concern that this would break the

BuildScansPerformanceTest, but that test doesn't actually use the

BaselineVersion class at all.

    • -4
    • +0
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 31 more files in changeset.
Revert "Make strict performance testing default."

This reverts commit 784d7476556e63c8d7f15919ae9875c3b7181aa3.

    • -0
    • +4
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 31 more files in changeset.
Make strict performance testing default.

- This makes it impossible to set maximum regression limits manually and

defaults to using a statistically derived allowable amount of

variance.

    • -4
    • +0
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 31 more files in changeset.
Allow specifying the list of baselines from command line

This commit introduces the ability to set the list of baseline versions for performance tests through a command-line

flag (`--baselines`) or a system property (`org.gradle.performance.baselines`). The list must be specified as a

comma-separated list of versions or as a semicolon separated list of versions.

Two versions are handled particularly:

* `last` corresponds to the last release of Gradle

* `nightly` corresponds to the latest build of Gradle (`master` branch)

This commit also removes the "ad-hoc" mode for executing tests. The idea is to replace it with this flag, which

can be set to `nightly`. However, the "ad-hoc" mode skipped writing results to the database, in order to avoid

polluting the performance results DB with tests. If you want to do this, you need to set the `gradle.performance.db.url`

to a local, temporary database:

```

./gradlew cleanSmallOldJava smallOldJava cleanPerformanceTest performanceTest --scenarios 'clean Java build smallOldJava (daemon)' -x prepareSamples --baselines nightly -Porg.gradle.performance.db.url=jdbc:h2:./build/database

```

    • -3
    • +2
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 10 more files in changeset.
Fix GC logging location in Maven performance tests

    • -1
    • +2
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 4 more files in changeset.
Execute performance test scenarios on a fresh working copy

Reusing whatever state the last test left behind can make performance

seem better (because of preexisting caches) or worse (because of lots of output).

This makes the results dependent on the order in which the tests are executed.

It also prevented us from using incremental build for the project templates.

We now create a fresh copy of the template project for each test run,

fixing both of these problems at once.

    • -0
    • +1
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 14 more files in changeset.
Extract performance test fixtures to separate project

    • -0
    • +56
    ./AbstractCrossVersionPerformanceTest.groovy
  1. … 248 more files in changeset.