DefaultCacheLockingManagerIntegrationTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Rename to ArtifactCacheLockingManager

The name CacheLockingManager suggested it was used to manage locking of

caches in general instead of just for the artifacts cache.

Resolves gradle/gradle-private#1338.

    • -294
    • +0
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 28 more files in changeset.
Delete empty parent dirs of deleted cache entries

Prior to this commit parent directories of deleted cache entries were

not checked whether they're eligible for deletion. Now, parent

directories that are now empty are checked up to the base directory.

For the artifact transforms cache the following directories marked with

(+) are now also deleted once all subdirectories (*) have been deleted:

- files-1.1

- artifact-a (+)

- hash 1 (*)

- hash 1.1

- ...

- hash 2 (*)

- ...

- ... (+)

For the artifact cache:

- modules-2

- files-2.1

- groupId (+)

- artifactId (+)

- version (+)

- hash1 (*)

- file

- hash2 (*)

- file

- resources-2.1

- 0 (+)

- hash 1 (*)

- hash 2 (*)

- ...

- ... (+)

Resolves gradle/gradle-private#1339.

    • -0
    • +4
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 3 more files in changeset.
Use inception time of journal as fallback

Prior to this commit the file modification timestamp was used as

fallback when the last access time was unknown. That lead to a lot of

files being deleted when first using the new Gradle distribution that

includes cleaning up of shared caches. In particular for dependencies

that have not been updated in the last 30 days and were not used in the

first build, lots of files would have to be redownloaded

This commit persists the inception time of the file access time journal

and uses it as the fallback for unknown files. This way, less files will

be cleaned up initially.

    • -8
    • +7
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 8 more files in changeset.
Fix integration tests

    • -2
    • +4
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 1 more file in changeset.
Require own user home for tests that delete the file access journal

    • -0
    • +2
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 1 more file in changeset.
Use separate cache for file access time journal

Storing a file access time journal inside each cache causes a lot of

contention in particular with caches that require a lot of exclusive

access like the artifact cache.

Instead, a new `journal-1` cache that is managed by a user-home-scoped

service is now used to keep track of file access times for all caches

that want to use it.

    • -29
    • +22
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 23 more files in changeset.
Add integration test for ArtifactResolutionQuery artifact redownloading

    • -0
    • +44
    ./DefaultCacheLockingManagerIntegrationTest.groovy
Add integration tests for redownloading deleted script/text resources

    • -0
    • +63
    ./DefaultCacheLockingManagerIntegrationTest.groovy
Improve test readability

    • -6
    • +4
    ./DefaultCacheLockingManagerIntegrationTest.groovy
Use constants for cache directory names for robust tests

    • -1
    • +2
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 3 more files in changeset.
Write access time asynchronously, read synchronously

File access times are now written asynchronously while the cache is

being used. When it's about to be closed, they are now read

synchronously from the cleanup action because the cache access worker

has then already been stopped.

    • -8
    • +9
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 31 more files in changeset.
Don't clean up metadata descriptors

Metadata descriptors are small (about 4 KB) and thus don't need to be

cleaned up.

    • -18
    • +3
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 4 more files in changeset.
Track artifact cache file access in PersistentIndexedCache

This commit introduces the `FileAccessJournal` interface and provides

two implementations:

ModificationTimeFileAccessJournal::

Reads and sets `File.lastModified()`.

IndexedCacheBackedFileAccessJournal::

Uses a PersistentIndexedCache to store the access timestamp.

The latter is now used in DefaultCacheLockingManager for the artifact

cache. All other PersistentCaches still use the former.

    • -26
    • +21
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 30 more files in changeset.
Mark cached files accessed when resolved via index

Both CachingModuleComponentRepository and

DefaultCacheAwareExternalResourceAccessor use an index to resolve

cached artifacts. Both now notify the underlying FileStore which now

implements FileAccessTracker. The new AccessTrackingFileStore interface

combines both interfaces so the GroupedAndNamedUniqueFileStore

subclasses (ArtifactIdentifierFileStore and ExternalResourceFileStore)

can be injected into dependent components.

    • -5
    • +62
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 32 more files in changeset.
Clean up least recently used entries in module metadata store

The files in the module metadata store cache are now automatically

cleaned up based on a least recently used strategy. The structure in

the cache is as follows:

- modules-2

- metadata-2.58

- descriptors

- groupId

- artifactId

- version

- hash1 (*)

- file

- hash2 (*)

- file

The cache now keeps track of files accessed inside the marked hash

directories. The cleanup action then deletes all such files that have

not been accessed in the last 30 days.

    • -1
    • +10
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 3 more files in changeset.
Clean up least recently used entries in artifacts file store

The files in the artifacts file store cache are now automatically

cleaned up based on a least recently used strategy. The structure in

the cache is as follows:

- modules-2

- files-2.1

- groupId

- artifactId

- version

- hash1 (*)

- file

- hash2 (*)

- file

The cache now keeps track of files accessed inside the marked hash

directories. The cleanup action then deletes all such files that have

not been accessed in the last 30 days.

    • -10
    • +37
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 15 more files in changeset.
Clean up least recently used entries in external resources file store

The files in the external resources file store cache are now

automatically cleaned up based on a least recently used strategy. The

structure in the cache is as follows:

- modules-2

- resources-2.1

- 0

- hash 1 (*)

- hash 2 (*)

- ...

- ...

The cache now keeps track of files accessed inside the marked hash

directories. The cleanup action then deletes all such files that have

not been accessed in the last 30 days.

    • -0
    • +114
    ./DefaultCacheLockingManagerIntegrationTest.groovy
  1. … 29 more files in changeset.