performance

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Automatically determine if `origin` should be used (#11580)

This commit changes how the `DetermineBaselines` task figures out

where to fetch sources. It was assuming a remote named `origin`,

but in practice it can be anything else, in particular `upstream`.

Now we will ask Git for the remotes and find the upstream one.

Pin all remote projects to exact refs, don't use branches

Pinning to branches means the build is not fully reproducible, as changes in the remote repo can alter the outcome of this build.

  1. … 3 more files in changeset.
Use dedicated user with token-based authentication to execute… (#11044)

Replace password-based authentication with token in distributed performance tests

  1. … 7 more files in changeset.
Use dedicated user with token-based authentication to execute… (#11044)

Replace password-based authentication with token in distributed performance tests

  1. … 7 more files in changeset.
Use dedicated user with token-based authentication to execute… (#11044)

Replace password-based authentication with token in distributed performance tests

  1. … 7 more files in changeset.
Remove `distributedFullPerformanceTest(s)`

  1. … 1 more file in changeset.
Add new experiment coordinator build

which runs once a week. The historical coordinator now only runs

regression tests.

  1. … 2 more files in changeset.
Run only slow regression tests every day

and maybe later on every commit on `master`. The experiments are

already run weekly as part of the `historical` performance tests.

  1. … 2 more files in changeset.
Move rerun strategy to property

and add some DSL to configure it.

  1. … 4 more files in changeset.
Remove unused code

  1. … 2 more files in changeset.
New performance process (#10361)

This PR introduces new performance test process. See more details in https://docs.google.com/document/u/1/d/1pghuxbCR5oYWhUrIK2e4bmABQt3NEIYOOIK4iHyjWyQ/edit#heading=h.is4fzcbmxxld

  1. … 50 more files in changeset.
Prefer composition over inheritance in PerformanceTest hierarchy (#10262)

### Context

Previously, we had a bad `PerformanceTest` hierarchy:

- PerformanceTest

- ReportGenerationPerformanceTest

- BuildScanPerformanceTest

- DistributedPerformanceTest

- RerunDistributedPerformanceTest

This PR does a refactoring - extracts a `PerformanceTestReporter`/`DistributedPerformanceTestReporter` out of the `PerformanceTest` hierarchy, because https://github.com/gradle/gradle-private/issues/2192 wants a report, too.

The hierarchy after the refactoring:

- PerformanceTest

- DistributedPerformanceTest

- PerformanceTestReporter

So we can make things easier.

  1. … 10 more files in changeset.
Add missing type information to `PerformanceTestPlugin`

For compatibility with Kotlin 1.3.41.

Change all subprojects to use 'implementation' dependencies

This includes:

- All projects now explicitly declare all dependencies to other

subprojects. This makes issues more visible, guards for accidental

addition of new dependencies, and leaks much less transitive

dependencies on the compile classpathes.

- All usages of 'runtime' to declare dependencies have been replaced

with 'runtimeOnly'

- All projects are now `java-library` (and declare this explicitly)

- Most remaining Groovy scripts are translated to Kotlin

- The old 'compile' and 'runtime' configurations are not

configured/created anymore for the 'testFixture' and 'integTest'

source sets.

- Some obsolete dependencies (see previous commits) are removed

- 'api' is used scarcely on purpose as the current project structure is

not well designed for this. The projects contain code for several

concerns of the build tool and thus putting complete projects on the

API of others exposes too much. They should be split up along

functional concerns first.

  1. … 112 more files in changeset.
Rerun distributed performance test in RERUNNER step (#8801)

After the improvement of automatically rerunning and tagging, we want to manage performance test in the same way:

- Only run each performance test scenario once.

- If it fails, `GRADLE_RERUNNER` will kick in and rerun the failed scenario. The good thing is that it might be scheduled to another build agent, which mitigates the effect of bad agent.

This PR does:

- Remove all `Retry` from performance tests.

- Add `GRADLE_RERUNNER` to performance tests and refactor some code.

- Add tests for `PerformanceTest`.

- Since `GRADLE_RERUNNER` depends on reading of test binary result, write binary test result file in `RerunableDistributedPerformanceTest`.

  1. … 23 more files in changeset.
Fix cache-hit in flakiness detection performance test (#8482)

We don't want build cache hit in performance flakiness detection, however, previously the coordinator build resolves "flakiness-detection-commit" baseline to real commit id "5.3-commit-237a600", resulting in unexpected cache hit.

This PR fixes it by:

- On coordinator's side, pass "flakiness-detection-commit" as it is to worker build.

- On worker's side, worker build resolves "flakiness-detection-commit" to real commit version - this disables build cache.

Since `DetermineBaselines` is becoming more and more complex, this PR also adds a unit test for `DetermineBaselines` class.

  1. … 3 more files in changeset.
Detect flaky performance test scenarios (#8367)

As part of https://github.com/gradle/gradle-private/issues/1635 , we want to detect flaky performance test with a weekly job, in order to know which scenarios are flaky.

  1. … 21 more files in changeset.
Fetch branch name from environment variable when necessary (#7947)

Previously we use JGit's branch, which might be not accurate.

Now we prefer environment variable over JGit.

  1. … 3 more files in changeset.
Build commit distribution when necessary (#7616)

This fixes https://github.com/gradle/gradle-private/issues/1640 and partially fixes https://github.com/gradle/gradle-private/issues/1632 .

In https://github.com/gradle/gradle/pull/7337 we supported running performance tests against fork point commit, now we want to make `5.1-commit-1a2b3c4d` a valid baseline version - users can specify this version in `--baselines` parameter:

- If commit baseline is set explicitly (like `5.1-commit-1a2b3c4d`), then build the corresponding commit distribution.

- Otherwise, if the current branch is not `master` or `release`, calculate the fork point commit then build the corresponding commit distribution.

This PR also publishes build scan to TeamCity build log and reuses `RemoteProject` logic and get rid of duplicate `git clone` code.

    • -0
    • +97
    ./BuildCommitDistribution.kt
    • -0
    • +75
    ./DetermineBaselines.kt
  1. … 3 more files in changeset.
Remove empty performanceReport task

Fix deprecations in buildSrc

Signed-off-by: Paul Merlin <paul@gradle.com>

  1. … 2 more files in changeset.
Run performance test against fork point commit (#7337)

This closes https://github.com/gradle/gradle-private/issues/1526

Basically, the idea is, we should run feature branch performance tests against the distribution which is built from the "fork point commit". This can reduce the effects of performance regression which is not introduced by the feature branch.

Two tasks are added into the build and depended by performance task:

- `determineForkPoint`, run `git` command to determine the fork point commit. It's done during the execution phase, then it sets the commit for `buildForkPointDistribution` task.

- `buildForkPointDistribution`, try to build a distribution from the fork point commit.

`buildForkPointDistribution` is cachable so it only needs to run once. Its input is the fork point commit, the output is two files: a binary distribution and a tooling api jar. The generated binary distribution is used to run individual worker tests.

    • -0
    • +104
    ./BuildForkPointDistribution.kt
    • -0
    • +45
    ./DetermineForkPoint.kt
  1. … 10 more files in changeset.
Let the Gradle build work with a disallowed method

Until this can be fixed in https://github.com/gradle/gradle-native/issues/864

Let the Gradle build work with a disallowed method

Until this can be fixed in https://github.com/gradle/gradle-native/issues/864

Avoid the use of configureEach inside a lazy task's configuration action

Fix broken buildScanPerformance tests (#6814)

* Fix broken buildScanPerformance tests

* Fix BuildScanPerformanceTest's reportDir

* Decrease to 1 for now to test

* Fix warning

* Change test name to test id

* Make BuildScanPerformanceTest cacheable

* Make ReportGenerationPerformanceTest compile static

* Fix override issue

  1. … 5 more files in changeset.
Merge branch 'master'

Improve performance report generation (#6642)

This PR improves performance report by:

- Merge different reports into one.

- Sort test scenarios in the order:

- Failed scenarios.

- Scenarios which are about to fail.

- Confidence.

- Show all important information in relevant execution.

- Show different result in different colors.

  1. … 16 more files in changeset.
And even more memory for JFR

Merge branch 'master'

  1. … 4 more files in changeset.