org

- Replace AtomicReference with MutableReference

- Replace AtomicBoolean with MutableBoolean

`UnusedVersionsCacheCleanup` now extends `AbstractCacheCleanup` to reuse

its timeout checking and to make it usable as a `CleanupAction` for a

`PersistentCache`. `CompositeCleanupAction` now allows to use multiple

top-level `CleanupActions`.

Prior to this commit parent directories of deleted cache entries were

not checked whether they're eligible for deletion. Now, parent

directories that are now empty are checked up to the base directory.

For the artifact transforms cache the following directories marked with

(+) are now also deleted once all subdirectories (*) have been deleted:

- files-1.1

- artifact-a (+)

- hash 1 (*)

- hash 1.1

- ...

- hash 2 (*)

- ...

- ... (+)

For the artifact cache:

- modules-2

- files-2.1

- groupId (+)

- artifactId (+)

- version (+)

- hash1 (*)

- file

- hash2 (*)

- file

- resources-2.1

- 0 (+)

- hash 1 (*)

- hash 2 (*)

- ...

- ... (+)

Resolves gradle/gradle-private#1339.

When a file is deleted by the LeastRecentlyUsedCacheCleanup action, the

corresponding entry is now removed from the used FileAccessTimeJournal

in order to avoid storing obsolete entries that would never get removed.

Storing a file access time journal inside each cache causes a lot of

contention in particular with caches that require a lot of exclusive

access like the artifact cache.

Instead, a new `journal-1` cache that is managed by a user-home-scoped

service is now used to keep track of file access times for all caches

that want to use it.

File access times are now written asynchronously while the cache is

being used. When it's about to be closed, they are now read

synchronously from the cleanup action because the cache access worker

has then already been stopped.

Signed-off-by: Rodrigo B. de Oliveira <rodrigo@gradle.com>

This commit introduces the `FileAccessJournal` interface and provides

two implementations:

ModificationTimeFileAccessJournal::

Reads and sets `File.lastModified()`.

IndexedCacheBackedFileAccessJournal::

Uses a PersistentIndexedCache to store the access timestamp.

The latter is now used in DefaultCacheLockingManager for the artifact

cache. All other PersistentCaches still use the former.

The files in the artifacts file store cache are now automatically

cleaned up based on a least recently used strategy. The structure in

the cache is as follows:

- modules-2

- files-2.1

- groupId

- artifactId

- version

- hash1 (*)

- file

- hash2 (*)

- file

The cache now keeps track of files accessed inside the marked hash

directories. The cleanup action then deletes all such files that have

not been accessed in the last 30 days.

The files in the external resources file store cache are now

automatically cleaned up based on a least recently used strategy. The

structure in the cache is as follows:

- modules-2

- resources-2.1

- 0

- hash 1 (*)

- hash 2 (*)

- ...

- ...

The cache now keeps track of files accessed inside the marked hash

directories. The cleanup action then deletes all such files that have

not been accessed in the last 30 days.

Prior to this commit, the check whether initialization is required

threw a `FileNotFoundException` when cache.properties did not exist.

Since the file is only created when `DefaultPersistentDirectoryCache`

is used instead of `DefaultPersistentDirectoryStore`, this can happen

when re-using an existing cache directory that used to be accessed

using the latter but from now on will be accessed using the former.

For instance, when adding a cleanup action, `DefaultCacheFactory` will

no longer return a plain store but a Ccache.

The missing properties file will now be ignored when the cache is not

using any properties. Otherwise, it will cause the cache to be

re-initialized, i.e. all contained files will be deleted.

In addition, this commit fixes a `NullPointerException` that occurred

when a property was present in the cache's new properties but missing

in the existing cache.properties file.

The files in the transforms cache are now automatically cleaned up based

on a least recently used strategy. The structure in the cache is as

follows:

- files-1.1

- artifact-a

- hash 1 (*)

- hash 1.1

- ...

- hash 2 (*)

- ...

- ...

- metadata-1.1

The cache now keeps track of files accessed inside the first hash (*)

level by touching the corresponding directory. The cleanup action then

deletes all such directories that have not been accessed in the last 7

days.

Issue: #1085

Should reduce garbage creation a little.

Signed-off-by: Lóránt Pintér <lorant@gradle.com>

Previously '.part' files were left behind after local cache failures, because we expected they would be useful in debugging these cases. That never happened, so we are now removing them even if there's an error, and also during garbage collection.

Signed-off-by: Lóránt Pintér <lorant@gradle.com>

- Introduce a new CleanupAction

- Decorate CleanupActions via CleanupActionFactory

- Remove the special case for keeping .part files

- Ask PersistentCache for "reserved" files, these files should never be cleaned up

- Reduce the number of build operations produced when cleaning a PersistentCache

Signed-off-by: Sterling Greene <sterling@gradle.com>

Signed-off-by: Sterling Greene <sterling@gradle.com>

This replaces Guava's hashing. We need this for two reasons:

- it allows the addition of custom hash functions, like BLAKE2b which would not be possible with Guava's closed design

- it fixes a few performance issues that Guava only fixes in version 20 (which we can't upgrade to right now, because we still need to support Java 5 in places)

If many processes fight and wait for the same lock, the timeout in one

process could be triggered, because it is "last in line". All the times

others hold the lock are added up for the timeout.

This change resets the timeout as soon as the lock owner changes.

The action is now also responsible for receiving confirmation that the

lock is in the process of being released.

+review REVIEW-6562