DefaultFileSystemSnapshotterTest.groovy

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move snapshotting files to own package

    • -300
    • +0
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 134 more files in changeset.
Remove FileSystemSnapshotter.getContentHash

This changes the path sensitivity for artifact transforms to

relative.

    • -44
    • +0
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 10 more files in changeset.
Rename snapshotAll

    • -12
    • +12
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 11 more files in changeset.
Merge file and directory cache

Instead of using two different caches for file and directory snapshots,

there is only one. We now cache the metadata of individual files

separately.

    • -14
    • +14
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 32 more files in changeset.
Pass directory to postVisitDirectory

    • -3
    • +3
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 15 more files in changeset.
Rename Snapshot to Hashable

    • -10
    • +3
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 16 more files in changeset.
Separate fingerprints and snapshots (#5927)

This de-tangles FileContentSnapshot (and removes it) from NormalizedFileSnapshots.

Also it makes clear that only PhysicalSnapshots can be used to check if a file changed the timestamp.

    • -20
    • +22
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 87 more files in changeset.
Calculate Merkle hashes for PhysicalSnapshots

    • -3
    • +4
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 21 more files in changeset.
Retain hierarchical structure in file system mirror (#5844)

The `FileSystemSnapshotter` now returns a root for each snapshotted file, possible containing all the children in hierarchy as the directory would no the file system.

This PR also contains some performance related changes:

- Stop interning path segments: That was only necessary when we stored the same path segments in many RelativePath objects. We don't do this any more, so interning should not be necessary and make things faster.

Actually, up-to-date assemble on largeMonolithicJavaProject (parallel false)

is about 40ms faster with this change (2.5 %).

- Don't use IndexedNormalizedFileSnapshots: It seems like there are no real performance gains by using IndexedNormalizedFileSnapshots.

    • -30
    • +120
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 131 more files in changeset.
Split file content caches into mutable and immutable files

Store content information about immutable files in the user home.

This means that things like external dependencies or the gradleApi

jar will only be analyzed once and that this analysis can then be

reused by other builds. This should improve build times after clean

checkouts and will also speed up our own integration tests.

    • -1
    • +1
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 16 more files in changeset.
Expand current test coverage

    • -0
    • +135
    ./DefaultFileSystemSnapshotterTest.groovy
Fix non-detected stale output file removal (#4264)

    • -3
    • +3
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 20 more files in changeset.
Replaced a direct usage of `FileSystemMirror` with a new method on `FileSystemSnapshotter`.

    • -0
    • +27
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 6 more files in changeset.
Make test case easier to understand

    • -8
    • +7
    ./DefaultFileSystemSnapshotterTest.groovy
Reuse cached snapshots of unfiltered trees

It is quite common for one task to generate some output

and another task to take a filtered version of that as an

input. Up until now we would snapshot the directory again,

even though the generating task put a snapshot of its full

output into the in-memory cache.

Instead, we now check whether we already have a snapshot of

the unfiltered contents of a directory and filter it in memory

instead of walking the file system again. This significantly

reduces up-to-date build times for Android projects, since

the Android plugin uses plenty of filtered trees.

    • -0
    • +26
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 3 more files in changeset.
Fix merge issues

    • -1
    • +1
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 22 more files in changeset.
Move FileHasher to base-services

+review REVIEW-6562

    • -1
    • +1
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 74 more files in changeset.
Hash script sources separately

+review REVIEW-6562

    • -2
    • +2
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 22 more files in changeset.
Avoid snapshotting a given file or directory from multiple threads at the same time.

Also moved responsibility for hashing a file/directory and caching the result in-memory from `DefaultTransformedFileCache` to `FileSystemSnapshotter` and friends, so that this is available outside of artifact transform caching and so that there is a single place that makes the decision for whether information for a given file/directory can be reused or should be calculated again.

    • -15
    • +66
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 16 more files in changeset.
Reduced the memory required for `FileSnapshot` instances.

    • -2
    • +2
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 12 more files in changeset.
Fixed typo in method name.

    • -5
    • +5
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 3 more files in changeset.
Changed file snapshotting to assume that files and directories contained in the downloaded artifact and artifact transform output caches do not change during a build, and so to make snapshotting faster.

File system state cached in-memory during a build for these files and directories is not discarded when a task action runs. This avoids scanning the file system potentially many times during a build over files that have almost certainly not changed. All other file system state is still discarded. This strategy is intentionally very simple and can be improved later.

This change also means that jars from the artifact transform output cache are not copied into the jar cache when included in a classpath from which Gradle needs to create a `ClassLoader`. This was already the case for jars from the downloaded artifacts cache.

    • -1
    • +2
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 13 more files in changeset.
Extracted `FileSystemSnapshotter` service out of `AbstractFileCollectionSnapshotter`.

The `FileSystemSnapshotter` takes care of snapshotting the current state of a file or directory in the file system and the associated in-memory mirroring.

The `FileCollectionSnapshotter` implementations then take care of transforming these snapshots to arrive at the snapshot for a given file collection when that file collection is used for a particular purpose.

    • -0
    • +154
    ./DefaultFileSystemSnapshotterTest.groovy
  1. … 16 more files in changeset.