Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Prefer loopback bind address if available for IPC

Keep file lock communicator bound to wildcard address

Signed-off-by: Paul Merlin <paul@gradle.com>

  1. … 6 more files in changeset.
Prefer loopback bind address if available for IPC

Keep file lock communicator bound to wildcard address

Signed-off-by: Paul Merlin <paul@gradle.com>

  1. … 6 more files in changeset.
Move in-memory caches to persistent-cache project

    • -0
    • +203
    ./gradle/cache/internal/InMemoryCacheDecoratorFactoryTest.groovy
    • -0
    • +195
    ./gradle/cache/internal/InMemoryDecoratedCacheTest.groovy
    • -0
    • +47
    ./gradle/cache/internal/LoggingEvictionListenerTest.groovy
  1. … 48 more files in changeset.
Simplify TaskHistoryStore API

Replaced two similar methods with one, and made PersistentIndexedCacheParameters immutable and nicer to instantiate.

  1. … 21 more files in changeset.
Remove --recompile-scripts option

The corresponding `StartParameter` property is now deprecated.

Resolves #6306.

    • -36
    • +11
    ./gradle/cache/internal/DefaultPersistentDirectoryCacheTest.groovy
  1. … 21 more files in changeset.
Remove --recompile-scripts option

The corresponding `StartParameter` property is now deprecated.

Resolves #6306.

    • -36
    • +11
    ./gradle/cache/internal/DefaultPersistentDirectoryCacheTest.groovy
  1. … 21 more files in changeset.
Fix DefaultCacheAccessTest

Merge pull request #6150 from gradle/marc/fixes/1412-contention-handling-improvements

Improve file lock contention handling

Handle missing type correctly

When communicating with an old version, the length of the received

packet will be shorter than the byte array. Now, `FileLockPacketPayload`

only reads the type when it has been received and returns UNKNOWN if it

has not. Previously, it would have worked by accident because the last

byte was zero which corresponds to the ordinal number of UNKNOWN.

  1. … 2 more files in changeset.
Make file lock packet type explicit

`FileLockCommunicator` now includes an additional byte that represents

the `FileLockPacketType` when sending packets to other processes.

`DefaultFileLockContentionHandler` now checks for the

`LOCK_RELEASE_CONFIRMATION` type instead of interpreting any packet for

a lock for which it already has received a packet as a lock release

confirmation.

Adding the additional byte will be ignored by old versions which only

read the first 9 bytes. New versions can still read packets from old

versions that do not include the byte. They use the `UNKNOWN` type in

this case. Thus, this protocol change is backwards compatible.

  1. … 4 more files in changeset.
Ensure file lock is released before cache cleanup

Caches with on-demand locking keep hold of file locks unless contended

by other Gradle processes. Prior to this commit, such previously

acquired file locks were held for the complete duration of cache

cleanup only to be released immediately afterwards. While cache cleanup

was running, requests to release such file locks from other processes

(via `DefaultFileLockContentionHandler`) were not processed because the

`stateLock` of `DefaultCacheAccess` was already locked by another

thread and thus, the `ContentionAction` in

`LockOnDemandCrossProcessCacheAccess` was blocked until the `close()`

method in `DefaultCacheAccess` was finished.

Since cache cleanup only deletes files that are no longer in use, it

does not use file locking (which would not work anyway because the

current cache access infrastructure is not usable while the cache is

already being closed). Thus, this commit closes the

`CrossProcessCacheAccess` of the `DefaultCacheAccess` *before* cleaning

up caches.

Issue: gradle/gradle-private#1412

  1. … 1 more file in changeset.
Improve file lock contention handling

This commit improves the likelihood for lock requesters to acquire a

file lock after it is released due to contention. After the lock has

been released, the former lock holder now sends a packet to the sockets

of all requesters. While old clients will simply ignore the additional

packet, new clients will interpret it as a signal that the file lock has

been released and will try to acquire it immediately.

Issue: gradle/gradle-private#1412.

    • -0
    • +74
    ./gradle/cache/internal/DefaultFileLockManagerAwaitableFileLockReleasedSignalTest.groovy
    • -35
    • +56
    ./gradle/cache/internal/LockOnDemandCrossProcessCacheAccessTest.groovy
  1. … 10 more files in changeset.
Report deleted and skipped cache entries

This commit introduces `CleanupProgressMonitor` and a default

`ProgressLogger`-based implementation that tracks the total number of

deleted and skipped cache entries. `CleanupAction.clean()` now takes a

monitor as an additional argument. `DefaultPersistentDirectoryStore`

creates the default implementation and passes it to its `CleanupAction`.

    • -0
    • +58
    ./gradle/cache/internal/DefaultCleanupProgressMonitorTest.groovy
  1. … 10 more files in changeset.
Use ProgressLogger so cache cleanup is visible

    • -12
    • +14
    ./gradle/cache/internal/DefaultPersistentDirectoryCacheTest.groovy
    • -1
    • +2
    ./gradle/cache/internal/DefaultPersistentDirectoryStoreConcurrencyTest.groovy
  1. … 7 more files in changeset.
Make cache cleanup non-incremental

  1. … 10 more files in changeset.
Avoid reinitializing DefaultPersistentDirectoryStores

Prior to this commit, adding a `CleanupAction` when building a

`PersistentCache` using `CacheBuilder.withCleanup()` caused the used

implementation class to be changed from

`DefaultPersistentDirectoryStore` to `DefaultPersistentDirectoryCache`.

The latter adds initialization logic and has a very strict check in

place that verifies whether the lock file has been unlocked cleanly. If

not, it will delete all files in the cache in order to reinitialize it.

Since the mere addition of a cleanup action should not change such a

fundamental behavior, this commit moves the cleanup logic to the

`DefaultPersistentDirectoryStore` class and instantiates it when a

cleanup action is configured but no initialization related properties.

    • -1
    • +1
    ./gradle/cache/internal/DefaultPersistentDirectoryStoreConcurrencyTest.groovy
  1. … 4 more files in changeset.
Rename GradleVersionProvider to UsedGradleVersions

  1. … 11 more files in changeset.
Handle multi-component cache versions

- Let `UnusedVersionsCacheCleanup` handle cache versions with multiple

components, e.g. metadata-2.58.

- Introduce `CacheVersion` to handle formatting and parsing in one

place.

    • -0
    • +84
    ./gradle/cache/internal/CacheVersionTest.groovy
  1. … 10 more files in changeset.
Stop using atomic types in non-thread-safe contexts (#5813)

- Replace AtomicReference with MutableReference

- Replace AtomicBoolean with MutableBoolean

  1. … 17 more files in changeset.
Implement cleanup of shared versioned caches

`UnusedVersionsCacheCleanup` now extends `AbstractCacheCleanup` to reuse

its timeout checking and to make it usable as a `CleanupAction` for a

`PersistentCache`. `CompositeCleanupAction` now allows to use multiple

top-level `CleanupActions`.

    • -0
    • +122
    ./gradle/cache/internal/CacheVersionMappingTest.groovy
    • -0
    • +85
    ./gradle/cache/internal/UnusedVersionsCacheCleanupTest.groovy
  1. … 7 more files in changeset.
Delete empty parent dirs of deleted cache entries

Prior to this commit parent directories of deleted cache entries were

not checked whether they're eligible for deletion. Now, parent

directories that are now empty are checked up to the base directory.

For the artifact transforms cache the following directories marked with

(+) are now also deleted once all subdirectories (*) have been deleted:

- files-1.1

- artifact-a (+)

- hash 1 (*)

- hash 1.1

- ...

- hash 2 (*)

- ...

- ... (+)

For the artifact cache:

- modules-2

- files-2.1

- groupId (+)

- artifactId (+)

- version (+)

- hash1 (*)

- file

- hash2 (*)

- file

- resources-2.1

- 0 (+)

- hash 1 (*)

- hash 2 (*)

- ...

- ... (+)

Resolves gradle/gradle-private#1339.

  1. … 3 more files in changeset.
Improve test readability

  1. … 2 more files in changeset.
Delete access times for deleted files from journal

When a file is deleted by the LeastRecentlyUsedCacheCleanup action, the

corresponding entry is now removed from the used FileAccessTimeJournal

in order to avoid storing obsolete entries that would never get removed.

  1. … 6 more files in changeset.
Use separate cache for file access time journal

Storing a file access time journal inside each cache causes a lot of

contention in particular with caches that require a lot of exclusive

access like the artifact cache.

Instead, a new `journal-1` cache that is managed by a user-home-scoped

service is now used to keep track of file access times for all caches

that want to use it.

  1. … 23 more files in changeset.
Only query cache's reserved files once per cleanup

    • -0
    • +41
    ./gradle/cache/internal/NonReservedFileFilterTest.groovy
  1. … 3 more files in changeset.
Enable incremental cache cleanup

Instead of computing the complete list of eligible files at once,

SingleDepthFileFinder now returns an Iterable that creates an

Iterator that will incrementally walk the file tree.

In addition, the CleanupAction.clean() operation now takes a

CountdownTimer and implementations periodically check whether the timer

has expired. If so, the current cleanup will be aborted.

DefaultPersistentDirectoryCache now uses a timeout of 20 seconds which

is well below the file locking timeout of 60 seconds.

The marker file (gc.properties) will only be updated if the complete

cleanup was able to finish without expiring the timeout.

    • -0
    • +81
    ./gradle/cache/internal/CompositeCleanupActionTest.groovy
  1. … 9 more files in changeset.
Write access time asynchronously, read synchronously

File access times are now written asynchronously while the cache is

being used. When it's about to be closed, they are now read

synchronously from the cleanup action because the cache access worker

has then already been stopped.

    • -68
    • +0
    ./gradle/cache/internal/IndexedCacheBackedFileAccessJournalTest.groovy
  1. … 30 more files in changeset.
Let `CacheKeyBuilder` accept `HashCode` components

Signed-off-by: Rodrigo B. de Oliveira <rodrigo@gradle.com>

  1. … 2 more files in changeset.
Track artifact cache file access in PersistentIndexedCache

This commit introduces the `FileAccessJournal` interface and provides

two implementations:

ModificationTimeFileAccessJournal::

Reads and sets `File.lastModified()`.

IndexedCacheBackedFileAccessJournal::

Uses a PersistentIndexedCache to store the access timestamp.

The latter is now used in DefaultCacheLockingManager for the artifact

cache. All other PersistentCaches still use the former.

    • -0
    • +68
    ./gradle/cache/internal/IndexedCacheBackedFileAccessJournalTest.groovy
    • -0
    • +72
    ./gradle/cache/internal/LeastRecentlyUsedCacheCleanupTest.groovy
  1. … 28 more files in changeset.
Return Collection instead of File[]

  1. … 4 more files in changeset.