performance

This commit changes how the `DetermineBaselines` task figures out

where to fetch sources. It was assuming a remote named `origin`,

but in practice it can be anything else, in particular `upstream`.

Now we will ask Git for the remotes and find the upstream one.

Pinning to branches means the build is not fully reproducible, as changes in the remote repo can alter the outcome of this build.

Replace password-based authentication with token in distributed performance tests

and maybe later on every commit on `master`. The experiments are

already run weekly as part of the `historical` performance tests.

and add some DSL to configure it.

This PR introduces new performance test process. See more details in https://docs.google.com/document/u/1/d/1pghuxbCR5oYWhUrIK2e4bmABQt3NEIYOOIK4iHyjWyQ/edit#heading=h.is4fzcbmxxld

### Context

Previously, we had a bad `PerformanceTest` hierarchy:

- PerformanceTest

- ReportGenerationPerformanceTest

- BuildScanPerformanceTest

- DistributedPerformanceTest

- RerunDistributedPerformanceTest

This PR does a refactoring - extracts a `PerformanceTestReporter`/`DistributedPerformanceTestReporter` out of the `PerformanceTest` hierarchy, because https://github.com/gradle/gradle-private/issues/2192 wants a report, too.

The hierarchy after the refactoring:

- PerformanceTest

- DistributedPerformanceTest

- PerformanceTestReporter

So we can make things easier.

For compatibility with Kotlin 1.3.41.

This includes:

- All projects now explicitly declare all dependencies to other

subprojects. This makes issues more visible, guards for accidental

addition of new dependencies, and leaks much less transitive

dependencies on the compile classpathes.

- All usages of 'runtime' to declare dependencies have been replaced

with 'runtimeOnly'

- All projects are now `java-library` (and declare this explicitly)

- Most remaining Groovy scripts are translated to Kotlin

- The old 'compile' and 'runtime' configurations are not

configured/created anymore for the 'testFixture' and 'integTest'

source sets.

- Some obsolete dependencies (see previous commits) are removed

- 'api' is used scarcely on purpose as the current project structure is

not well designed for this. The projects contain code for several

concerns of the build tool and thus putting complete projects on the

API of others exposes too much. They should be split up along

functional concerns first.

After the improvement of automatically rerunning and tagging, we want to manage performance test in the same way:

- Only run each performance test scenario once.

- If it fails, `GRADLE_RERUNNER` will kick in and rerun the failed scenario. The good thing is that it might be scheduled to another build agent, which mitigates the effect of bad agent.

This PR does:

- Remove all `Retry` from performance tests.

- Add `GRADLE_RERUNNER` to performance tests and refactor some code.

- Add tests for `PerformanceTest`.

- Since `GRADLE_RERUNNER` depends on reading of test binary result, write binary test result file in `RerunableDistributedPerformanceTest`.

We don't want build cache hit in performance flakiness detection, however, previously the coordinator build resolves "flakiness-detection-commit" baseline to real commit id "5.3-commit-237a600", resulting in unexpected cache hit.

This PR fixes it by:

- On coordinator's side, pass "flakiness-detection-commit" as it is to worker build.

- On worker's side, worker build resolves "flakiness-detection-commit" to real commit version - this disables build cache.

Since `DetermineBaselines` is becoming more and more complex, this PR also adds a unit test for `DetermineBaselines` class.

As part of https://github.com/gradle/gradle-private/issues/1635 , we want to detect flaky performance test with a weekly job, in order to know which scenarios are flaky.

Previously we use JGit's branch, which might be not accurate.

Now we prefer environment variable over JGit.

This fixes https://github.com/gradle/gradle-private/issues/1640 and partially fixes https://github.com/gradle/gradle-private/issues/1632 .

In https://github.com/gradle/gradle/pull/7337 we supported running performance tests against fork point commit, now we want to make `5.1-commit-1a2b3c4d` a valid baseline version - users can specify this version in `--baselines` parameter:

- If commit baseline is set explicitly (like `5.1-commit-1a2b3c4d`), then build the corresponding commit distribution.

- Otherwise, if the current branch is not `master` or `release`, calculate the fork point commit then build the corresponding commit distribution.

This PR also publishes build scan to TeamCity build log and reuses `RemoteProject` logic and get rid of duplicate `git clone` code.

Signed-off-by: Paul Merlin <paul@gradle.com>

This closes https://github.com/gradle/gradle-private/issues/1526

Basically, the idea is, we should run feature branch performance tests against the distribution which is built from the "fork point commit". This can reduce the effects of performance regression which is not introduced by the feature branch.

Two tasks are added into the build and depended by performance task:

- `determineForkPoint`, run `git` command to determine the fork point commit. It's done during the execution phase, then it sets the commit for `buildForkPointDistribution` task.

- `buildForkPointDistribution`, try to build a distribution from the fork point commit.

`buildForkPointDistribution` is cachable so it only needs to run once. Its input is the fork point commit, the output is two files: a binary distribution and a tooling api jar. The generated binary distribution is used to run individual worker tests.

Until this can be fixed in https://github.com/gradle/gradle-native/issues/864

* Fix broken buildScanPerformance tests

* Fix BuildScanPerformanceTest's reportDir

* Decrease to 1 for now to test

* Fix warning

* Change test name to test id

* Make BuildScanPerformanceTest cacheable

* Make ReportGenerationPerformanceTest compile static

* Fix override issue

This PR improves performance report by:

- Merge different reports into one.

- Sort test scenarios in the order:

- Failed scenarios.

- Scenarios which are about to fail.

- Confidence.

- Show all important information in relevant execution.

- Show different result in different colors.