AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Upgrade Groovy dependency to version 2.5.10

    • -2
    • +2
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 6 more files in changeset.
Test Gradle build with Groovy 2.5.10-SNAPSHOT

Use a nightly version of Gradle that disallows NotYetImplementedASTTransformation in Groovy compiler classloader due to static references to a JUnit class.

Check in and use a locally repackaged version of groovy-all with Groovy 2.5.10-SNAPSHOT.

    • -2
    • +2
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 7 more files in changeset.
Make TAPI performance tests profilable

    • -15
    • +9
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 4 more files in changeset.
Use repository mirrors in performance tests

    • -9
    • +17
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 10 more files in changeset.
Use tapi class loader to check if class is available

    • -1
    • +1
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
Make studio mockup tests fail gracefully on old versions

    • -1
    • +1
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
Add rule for setting the test method name as performance test id

    • -0
    • +1
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 10 more files in changeset.
Fixed qualified class name

gradle/performance#382

    • -1
    • +1
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
Make sure BuildAction is available before using it

If this test runs with an older Gradle version, BuildAction might

not be available. The test will now throw a ClassNotFoundExcpetion

in that case.

    • -0
    • +1
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
Simplify performance measurements

The many measurements that we injected into the build under test were

skewing our measurements to the point of making them unreliable or

unrealistic. We now only measure end to end build time. Here's a breakdown

with the rationale for removing each other measurement:

- configuration time: can be done by having a `gradle help` scenario instead

- execution time: the user does not care whether a long build is stuck in execution or configuration

- setup/teardown: was ill-defined anyway, basically total - configuration - execution

- JIT compile time: this is nothing we can influence and thus pointless to measure

- Memory usage: Was only measured at one point in the build, which doesn't tell us anything about

any problems at any other point in the build

- GC CPU time: If this increases we'd see it in total execution time

Generally, looking at the graphs has never pointed us directly at the problem, we always need to

profile anyway. So instead of skewing our measurements with lots of profling code, we should

instead use a dedicated profiling job to measure if we actually see a regression.

Memory usage can be tested indirectly by giving each scenario a reasonable amount of memory.

If memory usage rises above that reasonable limit, we'd see execution time rise, telling us about

the regression. Generally, we do not optimize for smallest memory usage, but for fastest execution

with reasonable memory overhead.

This change also removes all JVM tweaking and wait periods which we introduced in an attempt to

make tests more predictable and stable. These tweaks have not really helped us achieve more stable

tests and have often done the opposite. They also add lots of complexity and make our tests more

unrealistic. A real users will not add all these JVM options to Gradle.

    • -1
    • +1
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 59 more files in changeset.
Support customizing invocations in performance tests

- remove previous parameterized generics that lead to a dead end

    • -1
    • +1
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 13 more files in changeset.
Add ability for an Android performance test to customize the `BuildActionExecuter`

This commit allows a conventional build action to call back code in the performance test executor, to tweak the configuration. This is important

for performance tests that may need different JVM settings or system properties than the ones from the command line.

    • -2
    • +12
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 4 more files in changeset.
Simplify the way we specify the action to be used in perf test

    • -2
    • +2
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 2 more files in changeset.
Add missing header

    • -0
    • +15
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
Implement Android Studio Mockup performance test

This commit adds a new kind of cross-version performance test which simulates the behavior of Gradle under Android Studio.

The first simulation is based on project synchronization and relies on previous work found on the android-studio-sync-test

(https://github.com/adammurdoch/android-studio-sync-test) project.

It's worth noting that the classes in `internal-android-performance-testing` are collecting some statistics which are *not*

used when generating the reports. This commit just setups the basic infrastructure for performance testing of Android projects

in the context of Android Studio.

Some implementation notes:

- the custom model used when calling the Tooling API live in a dedicated project (`internalAndroidPerformanceTesting`) so that

we can generate a JAR that can in turn be added to the classpath when executing the test. This allows us to precompile the model

class and limit the impact of using Groovy to collect statistics.

- the `internalAndroidPerformanceTesting` project uses a trick to workaround an issue with some Android dependencies which

provide a `gradle-core` jar. This is problematic because our module lookup finds this jar first and tries to load it as the

main Gradle module, although it's not. For that reason, the dependencies are renamed (prefixed with `android-`) and added to

the classpath, avoiding this name clash.

- this classpath, as well as the generated custom model classes, are written in a `tapi-classpath.txt` file which is then

used by the performance test runner to set the tooling api classpath.

    • -0
    • +49
    ./AbstractAndroidStudioMockupCrossVersionPerformanceTest.groovy
  1. … 10 more files in changeset.