DefaultTransformedFileCacheTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Rename TransformedFileCache to CachingTransformerExecutor

    • -427
    • +0
    ./DefaultTransformedFileCacheTest.groovy
  1. … 14 more files in changeset.
Move exception handling to invoker

    • -25
    • +25
    ./DefaultTransformedFileCacheTest.groovy
  1. … 13 more files in changeset.
Rename TransformerRegistration -> Transformer

    • -13
    • +13
    ./DefaultTransformedFileCacheTest.groovy
  1. … 15 more files in changeset.
Simplify TransformOperation

so it only runs a step without including any further logic.

I.e. it is only used for generating build operations nothing else.

    • -13
    • +13
    ./DefaultTransformedFileCacheTest.groovy
  1. … 36 more files in changeset.
Some more changes

    • -58
    • +85
    ./DefaultTransformedFileCacheTest.groovy
  1. … 10 more files in changeset.
Drop Physical from Physical*Snapshot

    • -2
    • +2
    ./DefaultTransformedFileCacheTest.groovy
  1. … 40 more files in changeset.
Move snapshotting files to own package

    • -2
    • +2
    ./DefaultTransformedFileCacheTest.groovy
  1. … 134 more files in changeset.
Only hash name on demand

Always combining the Merkle hash and the name into a single hash for

artifact transforms is to expensive and causes a regression for

`get IDE model on largeAndroidBuild for Android Studio`.

    • -1
    • +1
    ./DefaultTransformedFileCacheTest.groovy
  1. … 1 more file in changeset.
Remove FileSystemSnapshotter.getContentHash

This changes the path sensitivity for artifact transforms to

relative.

    • -18
    • +23
    ./DefaultTransformedFileCacheTest.groovy
  1. … 10 more files in changeset.
Rename snapshotAll

    • -18
    • +18
    ./DefaultTransformedFileCacheTest.groovy
  1. … 11 more files in changeset.
Rename Snapshot to Hashable

    • -40
    • +18
    ./DefaultTransformedFileCacheTest.groovy
  1. … 16 more files in changeset.
FileCollectionFingerprint is not a Snapshot

    • -8
    • +22
    ./DefaultTransformedFileCacheTest.groovy
  1. … 5 more files in changeset.
Separate {Current,Historical}FileCollectionFingerprint

    • -2
    • +2
    ./DefaultTransformedFileCacheTest.groovy
  1. … 41 more files in changeset.
Make artifact and transforms caches user-home scoped

Both caches are now initialized and cleaned up in Gradle user home

scope so cleanup runs when the daemon is stopped.

    • -6
    • +1
    ./DefaultTransformedFileCacheTest.groovy
  1. … 5 more files in changeset.
Make dependency management cleanup actions observable

Wrap CleanupActions in build operations to make them observable in a

build session.

Resolves gradle/gradle-private#1350.

    • -2
    • +11
    ./DefaultTransformedFileCacheTest.groovy
  1. … 4 more files in changeset.
Track file access in AbstractCachedIndex

Instead of tracking access in DefaultCacheAwareExternalResourceAccessor

and CachingModuleComponentRepository this is now done in

AbstractCachedIndex which ensures it's done always when reading from

indexes of CachedItems and only does so when not already present in an

in-memory cache from a previous lookup.

    • -3
    • +17
    ./DefaultTransformedFileCacheTest.groovy
  1. … 39 more files in changeset.
Use separate cache for file access time journal

Storing a file access time journal inside each cache causes a lot of

contention in particular with caches that require a lot of exclusive

access like the artifact cache.

Instead, a new `journal-1` cache that is managed by a user-home-scoped

service is now used to keep track of file access times for all caches

that want to use it.

    • -2
    • +4
    ./DefaultTransformedFileCacheTest.groovy
  1. … 23 more files in changeset.
Write access time asynchronously, read synchronously

File access times are now written asynchronously while the cache is

being used. When it's about to be closed, they are now read

synchronously from the cleanup action because the cache access worker

has then already been stopped.

    • -56
    • +21
    ./DefaultTransformedFileCacheTest.groovy
  1. … 31 more files in changeset.
Clean up least recently used entries in artifact transforms cache

The files in the transforms cache are now automatically cleaned up based

on a least recently used strategy. The structure in the cache is as

follows:

- files-1.1

- artifact-a

- hash 1 (*)

- hash 1.1

- ...

- hash 2 (*)

- ...

- ...

- metadata-1.1

The cache now keeps track of files accessed inside the first hash (*)

level by touching the corresponding directory. The cleanup action then

deletes all such directories that have not been accessed in the last 7

days.

Issue: #1085

    • -4
    • +69
    ./DefaultTransformedFileCacheTest.groovy
  1. … 8 more files in changeset.
Fixed some incorrect spelling: MetaData -> Metadata

    • -2
    • +2
    ./DefaultTransformedFileCacheTest.groovy
  1. … 26 more files in changeset.
Make artifact transforms more efficient

Only create a build operation if we don't already

have a cached value in memory. Otherwise we end up

flooding our build operation infrastructure with

synchronization requests and also create lots of

garbage descriptor objects.

    • -1
    • +42
    ./DefaultTransformedFileCacheTest.groovy
  1. … 23 more files in changeset.
Use our own hashing architecture, take 2 (#2817)

This replaces Guava's hashing. We need this for two reasons:

- it allows the addition of custom hash functions, like BLAKE2b which would not be possible with Guava's closed design

- it fixes a few performance issues that Guava only fixes in version 20 (which we can't upgrade to right now, because we still need to support Java 5 in places)

    • -1
    • +1
    ./DefaultTransformedFileCacheTest.groovy
  1. … 159 more files in changeset.
Track task and task action types together with their classloaders

It looks a lot tidier this way.

+review REVIEW-6534

    • -1
    • +1
    ./DefaultTransformedFileCacheTest.groovy
  1. … 18 more files in changeset.
Avoid snapshotting a given file or directory from multiple threads at the same time.

Also moved responsibility for hashing a file/directory and caching the result in-memory from `DefaultTransformedFileCache` to `FileSystemSnapshotter` and friends, so that this is available outside of artifact transform caching and so that there is a single place that makes the decision for whether information for a given file/directory can be reused or should be calculated again.

    • -71
    • +40
    ./DefaultTransformedFileCacheTest.groovy
  1. … 16 more files in changeset.
Improved performance when searching for the result of an artifact transform.

Assume that files and directories that live in the downloaded artifact cache or artifact transform output cache are unlikely to change during a build, and so calculate the content hash for these files only once per build. A previous change removed the multiple scanning of these files, however, the calculation of the hash from the snapshot is currently still expensive, so do this only once per build.

Maintain a faster in-memory cache of transform -> result, in order to check whether the result still exists once per build.

    • -29
    • +119
    ./DefaultTransformedFileCacheTest.groovy
  1. … 6 more files in changeset.
Reworked how locking is applied around artifact transforms, so that 2 threads don't attempt to run the same transform when the previous output has been deleted.

    • -108
    • +42
    ./DefaultTransformedFileCacheTest.groovy
  1. … 9 more files in changeset.
Don't reuse the cached result of an artifact transform if the output no longer exists.

    • -17
    • +112
    ./DefaultTransformedFileCacheTest.groovy
  1. … 2 more files in changeset.
Mark unit test as `@UsesNativeServices`

    • -0
    • +2
    ./DefaultTransformedFileCacheTest.groovy
Reused the cached results of artifact transforms across build invocations.

    • -4
    • +22
    ./DefaultTransformedFileCacheTest.groovy
  1. … 3 more files in changeset.
Moved responsibility for calculation of the output directory for an artifact transfrom from `ArtifactTransformBackedTransformer` to `DefaultTransformedFileCache`, so that the input file snapshot and the resulting output directory are available to the caching logic.

    • -15
    • +46
    ./DefaultTransformedFileCacheTest.groovy
  1. … 8 more files in changeset.