Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Document lazy vs. realised ModuleComponentResolveMetadata

Also includes minor code refactoring to improve readability.

    • -0
    • +6
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 9 more files in changeset.
Rename method to better indicate meaning

The hash value returned by getOriginalContentHash() is the one from the

metadata parsed from a repository. It does not reflect any of the

potential mutations that happened to it at later stages.

The new name indicates this better.

  1. … 9 more files in changeset.
Improved name and documentation of attribute container serializer

    • -2
    • +2
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 10 more files in changeset.
Refactor ModuleResolveMetadata

Reorganize code in packages, move serialization related code closer to

classes to reduce need of public methods.

    • -462
    • +21
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 78 more files in changeset.
Refactor ModuleResolveMetadata

Reorganize code in packages, move serialization related code closer to

classes to reduce need of public methods.

    • -462
    • +21
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 78 more files in changeset.
Serialization of realised ModuleComponentResolveMetadata

This enables proper caching by making sure we can save and reload cache

values.

Fixes #5653

    • -4
    • +467
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 13 more files in changeset.
Serialization of realised ModuleComponentResolveMetadata

This enables proper caching by making sure we can save and reload cache

values.

Fixes #5653

    • -4
    • +467
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 13 more files in changeset.
Rename to ArtifactCacheLockingManager

The name CacheLockingManager suggested it was used to manage locking of

caches in general instead of just for the artifacts cache.

Resolves gradle/gradle-private#1338.

    • -3
    • +3
    ./artifacts/DefaultModuleArtifactCache.java
    • -5
    • +5
    ./artifacts/DefaultModuleArtifactsCache.java
    • -5
    • +5
    ./dynamicversions/DefaultModuleVersionsCache.java
  1. … 25 more files in changeset.
Realised metadata

This includes the following:

* Realising variants, by applying attributes, capabilities and

dependency rules

* Eagerly creating configurations and applying attributes, capabilites

and dependency rules

Issue #5653

    • -1
    • +21
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 17 more files in changeset.
Realised metadata

This includes the following:

* Realising variants, by applying attributes, capabilities and

dependency rules

* Eagerly creating configurations and applying attributes, capabilites

and dependency rules

Issue #5653

    • -1
    • +21
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 20 more files in changeset.
Normalize `ModuleIdentifier`

This commit reworks the `ComponentModuleIdentifier`/`ComponentModuleSelector`/`ModuleVersionSelector`

classes to use `ModuleIdentifier` under the hood, instead of storing denormalized strings. This has

the advantage that we can reduce the use of the module identifier factory, which is called very

often during dependency resolution. Sharing instances reduces the need for conversions, and makes

comparisons faster.

  1. … 164 more files in changeset.
Track file access in AbstractCachedIndex

Instead of tracking access in DefaultCacheAwareExternalResourceAccessor

and CachingModuleComponentRepository this is now done in

AbstractCachedIndex which ensures it's done always when reading from

indexes of CachedItems and only does so when not already present in an

in-memory cache from a previous lookup.

    • -2
    • +3
    ./artifacts/DefaultModuleArtifactCache.java
  1. … 38 more files in changeset.
Resurrect and fix DefaultModuleArtifactCacheTest

    • -0
    • +2
    ./artifacts/DefaultModuleArtifactCache.java
  1. … 2 more files in changeset.
Avoid resizing of linked hash set when we know the number of elements

Don't clean up metadata descriptors

Metadata descriptors are small (about 4 KB) and thus don't need to be

cleaned up.

  1. … 3 more files in changeset.
Track artifact cache file access in PersistentIndexedCache

This commit introduces the `FileAccessJournal` interface and provides

two implementations:

ModificationTimeFileAccessJournal::

Reads and sets `File.lastModified()`.

IndexedCacheBackedFileAccessJournal::

Uses a PersistentIndexedCache to store the access timestamp.

The latter is now used in DefaultCacheLockingManager for the artifact

cache. All other PersistentCaches still use the former.

  1. … 29 more files in changeset.
Mark cached files accessed when resolved via index

Both CachingModuleComponentRepository and

DefaultCacheAwareExternalResourceAccessor use an index to resolve

cached artifacts. Both now notify the underlying FileStore which now

implements FileAccessTracker. The new AccessTrackingFileStore interface

combines both interfaces so the GroupedAndNamedUniqueFileStore

subclasses (ArtifactIdentifierFileStore and ExternalResourceFileStore)

can be injected into dependent components.

  1. … 32 more files in changeset.
Clean up least recently used entries in module metadata store

The files in the module metadata store cache are now automatically

cleaned up based on a least recently used strategy. The structure in

the cache is as follows:

- modules-2

- metadata-2.58

- descriptors

- groupId

- artifactId

- version

- hash1 (*)

- file

- hash2 (*)

- file

The cache now keeps track of files accessed inside the marked hash

directories. The cleanup action then deletes all such files that have

not been accessed in the last 30 days.

  1. … 2 more files in changeset.
Create and wire in a ComponentMetadataRuleExecutor

This enables ComponentMetadataRule execution to be cached

Update the CrossBuildCachingRuleExecutor to work with a set of rule

instead of a single one. This means we cache or miss on a chain of rules

instead of single rules.

Fixes #5526

    • -0
    • +47
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 32 more files in changeset.
Add initial support component metadata supplier caching

This commit adds the infrastructure for caching of component metadata rules.

For now, the only kind of rule we can cache are the component metadata supplier

rules, which are used during dynamic version selection, but the executor

infrastructure can be reused to cache more kinds of rules.

The implementation relies on the principal that a "rule" can be safely cached if:

- the rule has no side effect, and uses the "class-based pattern"

- the implementation of the rule didn't change

- the inputs of the rule didn't change

For this, we introduce a new persistent cache (hence cross-build) with an in-memory

facing cache, that allows us to avoid the execution of a rule if the result is

already in the cache. The implementation makes use of _snapshotting_ to capture the

state of the inputs of a rule, which consists of the implementation of the rule and

its potential parameters. It's worth noting that at this stage we do not consider

the services the rule can use, it's going to be done in a subsequent commit.

This worked required a clarification of what an rule cares about (the result) in

opposition to what the user is faced with. For example, a component metadata supplier

rule output is a `ComponentMetadata`, but what we offer to the user, to build that

metadata, is a `ComponentMetadataSupplierDetails` instance. Similarly, component

metadata rules would have different kind of output and what the user manipulates.

The cache works with a primary key (for the first implemented cache, the key is the

module version identifier) and will invalidate the results based on the cache policy

(should refresh modules).

The persistent cache uses the snapshot as the key in the store. In theory, should we

consider that we have a nearly perfect hash function, we could instead use the hash

of the snapshot as the key. Measurements would have to be made to check if it's worth

implementing.

    • -0
    • +87
    ./SuppliedComponentMetadataSerializer.java
  1. … 46 more files in changeset.
Add initial support component metadata supplier caching

This commit adds the infrastructure for caching of component metadata rules.

For now, the only kind of rule we can cache are the component metadata supplier

rules, which are used during dynamic version selection, but the executor

infrastructure can be reused to cache more kinds of rules.

The implementation relies on the principal that a "rule" can be safely cached if:

- the rule has no side effect, and uses the "class-based pattern"

- the implementation of the rule didn't change

- the inputs of the rule didn't change

For this, we introduce a new persistent cache (hence cross-build) with an in-memory

facing cache, that allows us to avoid the execution of a rule if the result is

already in the cache. The implementation makes use of _snapshotting_ to capture the

state of the inputs of a rule, which consists of the implementation of the rule and

its potential parameters. It's worth noting that at this stage we do not consider

the services the rule can use, it's going to be done in a subsequent commit.

This worked required a clarification of what an rule cares about (the result) in

opposition to what the user is faced with. For example, a component metadata supplier

rule output is a `ComponentMetadata`, but what we offer to the user, to build that

metadata, is a `ComponentMetadataSupplierDetails` instance. Similarly, component

metadata rules would have different kind of output and what the user manipulates.

The cache works with a primary key (for the first implemented cache, the key is the

module version identifier) and will invalidate the results based on the cache policy

(should refresh modules).

The persistent cache uses the snapshot as the key in the store. In theory, should we

consider that we have a nearly perfect hash function, we could instead use the hash

of the snapshot as the key. Measurements would have to be made to check if it's worth

implementing.

    • -0
    • +87
    ./SuppliedComponentMetadataSerializer.java
  1. … 46 more files in changeset.
Intern strings when reading module metadata from cache

Dependency resolution of large dependency graphs involves a significant

number of comparisons of strings(group, artifact, version, ...). Most of

those come from the module metadata cache, and even if we use hashmaps,

we still need to perform `equals` comparisons on strings, when in most

of the cases they should be identical. This commit takes advantage of

knowing that to add a cost when we read module metadata (interning), but

realizing that the debt is paid when comparing strings during resolution.

The interner is build scoped (in order to avoid memory leaks), thread-safe,

and shared with the dependency notation converter, so that module selectors

created from strings found in the build scripts are using the same strings

as the ones from the module metadata cache.

Ideally, we should also do this for the strings used during parsing.

    • -0
    • +110
    ./StringDeduplicatingDecoder.java
  1. … 15 more files in changeset.
Serialize dependency attributes metadata

Now that module metadata may contain dependencies with attributes, we need to serialize them too.

This commit updates the component selector serializer, as well as the module metadata serializer,

to use this information, effectively bumping the cache metadata layout format version.

  1. … 18 more files in changeset.
Read dependency attributes in Gradle module metadata

This commit adds support for parsing dependency attributes in Gradle module metadata.

It is not yet making use of that information, it's only making sure that the parser

is able to parse and propagate it. It doesn't touch the serializer either, which is

going to be done in a separate commit.

Module metadata file version has been bumped to 0.4 as part of this work.

  1. … 17 more files in changeset.
Rename id accessors for consistency

Use `ComponentIdentifier getId()`

Use `ModuleVersionIdentifier getModuleVersionId()`

  1. … 65 more files in changeset.
Rename `CapabilitiesDescriptor` to `Capabilities`

... and `getCapababilitiesMetadata()` to `getCapabilities()` for consistency.

  1. … 36 more files in changeset.
Add support for capabilities in module metadata

This commit adds support for capabilities in module metadata. Capabilities are found under variants,

and we make sure to serialize the capabililities to their binary form too. However, there's still no

way to publish the capabilities, nor test fixtures to simulate published capabilities.

  1. … 9 more files in changeset.
Revert the initial implementation of capabilities

Revert "Remove the `Preference` inner class"

Revert "Add test case showing limitation of the current implementation of capabilities"

Revert "Report error whenever two modules in the graph disagree on a preference"

Revert "Support capabilities with project dependencies"

Revert "Consume capabilities from external modules"

Revert "Handle case of conflicting resolution in case multiple capabilities are found"

Revert "Make sure that conflict resolution on version still kicks in"

Revert "Wire capabilities handling into conflict resolution"

Revert "Introduce `CapabilitiesHandlerInternal`"

Revert "Validate module notations in capabilities"

Revert "Introduce the concept of "capabilities""

This reverts commit a6248b2c4e7a810051112fc2169c97d341b9f007.

This reverts commit 76e88732f403d1b295c879bdee0b0a92887e4173.

This reverts commit d9dbe0f6d909604d31857504133440f92996d3f3.

This reverts commit ac3ffa467dd73314b4526b79df3cedf6db4e6c13.

This reverts commit 97f523cd78cdca3819a8a6d65d33b72f8b972646.

This reverts commit d4c02e3f13340ce2d6114cd17f741ac51fee7796.

This reverts commit 4d4d71eb26a2828ff37fe700ae55530d9fff711d.

This reverts commit b0c962468a54affd3eaad8a5df698287d5a1ab4f.

This reverts commit 4030f642397f46330670b99cd6252e92684c53a1.

This reverts commit 80f39f5465990c870dac59b61ade4d8c68839fe2.

This reverts commit 8c0e02833303ea1a6a4e57af48278f24672c98ff.

  1. … 64 more files in changeset.
Support capabilities with project dependencies

Previously, it was possible to discover and solve conflicts between published modules, or through the `capabilities` block when

declaring capabilities for external modules. With this commit, it is now possible to declare capabilities in projects that are

used during conflict resolution.

  1. … 29 more files in changeset.
Consume capabilities from external modules

This commit introduces the ability to consume capabilities from external

modules. This is a pretty big change for multiple reasons:

- capabilities are no longer known beforehand (before resolution starts) but

can be discovered as we add more nodes to the graph

- conflict resolution cannot fail fast anymore, since it is possible for a

module to declare a capability that triggers a conflict with another module

in the graph, but a 3rd party module can express a preference

- capabilities are consumed from Gradle metadata files

- capabilities from the local component and capabilities from the graph are

merged during resolution

Capabilities are, in published metadata, available at the component level,

with the assumption that capabilities are the same for all variants. It is

not necessary for a module to express a preference for a capability, as

a 3rd party module can declare only a preference, for example.

  1. … 49 more files in changeset.