Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Remove extra serialized boolean and use real cache layout version

    • -2
    • +0
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 2 more files in changeset.
Introduce a checksum file cache service

This service is responsible for caching the checksums computed from

local file system. Because it's also used for dependency verification

writing and checking, this cache uses the existing infrastructure which

makes sure that if a file is updated locally, we expire the entry in

the cache.

This is done because there are lots of places in the code where we

used the legacy `HashUtil` class, which has no caching whatsoever.

It's, however, quite common to have a build which generates sha1

checksums multiple times for the same file. For example, during

publication.

    • -4
    • +4
    ./artifacts/AbstractArtifactsCache.java
    • -5
    • +6
    ./artifacts/DefaultCachedArtifact.java
    • -4
    • +4
    ./artifacts/DefaultCachedArtifacts.java
    • -6
    • +6
    ./artifacts/DefaultModuleArtifactCache.java
    • -2
    • +2
    ./artifacts/DefaultModuleArtifactsCache.java
    • -3
    • +3
    ./artifacts/InMemoryModuleArtifactCache.java
  1. … 92 more files in changeset.
Always serialize module sources

Module sources were only serialized in the cache metadata entry.

In practice, they belong to the module metadata, so they are now

properly serialized as part of it. This fixes the "force realize"

tests.

    • -47
    • +3
    ./ModuleMetadataCacheEntrySerializer.java
    • -0
    • +69
    ./ModuleSourcesSerializer.java
    • -20
    • +3
    ./PersistentModuleMetadataCache.java
  1. … 12 more files in changeset.
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

    • -20
    • +48
    ./ModuleMetadataCacheEntrySerializer.java
    • -3
    • +21
    ./PersistentModuleMetadataCache.java
  1. … 37 more files in changeset.
Revert "Remove redundant code"

This reverts commit a3b765a661bff56679846a3b75bd83870cea03f0.

    • -1
    • +35
    ./ModuleMetadataCacheEntrySerializer.java
Remove redundant code

Surprisingly, the whole `ModuleSource` serialization in module

metadata seems redundant. It looks like we always reconstruct

the module source from the various places so there's actually

no point in serializing.

    • -35
    • +1
    ./ModuleMetadataCacheEntrySerializer.java
Refactor `ModuleSource`

The `ModuleSource` concept was a bit messy. It was designed in order

to be able to store the origin of an artifact. Over time, it evolved

into storing more information, like snapshot timestamps, repositories

or content hash.

The code was convoluted because each part of the code was expecting

some kind of module source, but because of delegation, it wasn't

really possible to add/mix more sources.

This commit refactors this concept into a `ModuleSources` concept

which allows storing more information about a module source, in

a safe and consistent manner. No more wrapping/unwrapping, and each

code requiring a specific type of module source can query for it.

    • -3
    • +18
    ./ModuleMetadataCacheEntrySerializer.java
  1. … 60 more files in changeset.
Allow only virtual components as platform owners

The concept of n 'platform owner' for a node was introduced to deal with

virtual platforms, where no real node exists for a platform.

Since this is now the only case where this functionality is used,

we limit it to that.

    • -8
    • +7
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 8 more files in changeset.
Handle serialization for shadowed capabilities in the GMM case (#11179)

This is now done in the same way as it is done in the

AbstractRealisedModuleResolveMetadataSerializationHelper

for cases without GMM

Follow up to: #11118

  1. … 4 more files in changeset.
No more absolute path in metadata cache

Reference to file are now relative to the root of the module cache

instead of being absolute.

This allows moving the full module cache around to be reused with

a different root path.

The path are also normalized, so that they are portable across systems

where the name separator changes.

Issue #1338

    • -7
    • +50
    ./artifacts/DefaultModuleArtifactCache.java
  1. … 10 more files in changeset.
Fix a couple Java 8 warnings

    • -1
    • +1
    ./artifacts/DefaultModuleArtifactCache.java
  1. … 3 more files in changeset.
Rename inheritStrictVersions() -> endorseStrictVersions() (#10755)

This name change more clearly communicates the semantics of the

feature from a users point of view.

This commit also removes all mentions of 'inheriting' AND 'forSubgraph'.

There have been some leftovers in documentation/comments that

would have been misleading in the future. To make sure we catch all,

this also updates all variable/method/package names.

  1. … 70 more files in changeset.
Support dependency artifact selectors in Gradle Module Metadata

This is a compatibility feature for combining GMM and pure pom/ivy

modules.

  1. … 13 more files in changeset.
Introduce constraint inheritance API

  1. … 40 more files in changeset.
Add missing @Override to all modules

Signed-off-by: Paul Merlin <paul@gradle.com>

    • -0
    • +2
    ./ModuleMetadataCacheEntrySerializer.java
    • -0
    • +2
    ./artifacts/AbstractArtifactsCache.java
    • -0
    • +5
    ./artifacts/DefaultCachedArtifact.java
    • -0
    • +3
    ./artifacts/DefaultCachedArtifacts.java
    • -0
    • +6
    ./artifacts/DefaultModuleArtifactCache.java
    • -0
    • +4
    ./artifacts/DefaultModuleArtifactsCache.java
    • -0
    • +2
    ./dynamicversions/AbstractModuleVersionsCache.java
    • -0
    • +2
    ./dynamicversions/DefaultCachedModuleVersionList.java
    • -0
    • +4
    ./dynamicversions/DefaultModuleVersionsCache.java
    • -0
    • +1
    ./dynamicversions/DefaultResolvedModuleVersion.java
  1. … 985 more files in changeset.
Avoid the use of `Objects.hashCode`

This involves the creation of an array, which starts becoming

relevant when we create lots of those objects.

  1. … 3 more files in changeset.
Further deduplication of serialized metadata

Maven dependency metadata de-duplication now happens as well when

serializing untransformed metadata.

Fixes #8311

    • -6
    • +11
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 5 more files in changeset.
Deduplication of serialized metadata

When writing realised Maven metadata, when a cacheable

ComponentMetadataRule is used, a lot of duplication happened around

MavenDependencyDescriptor.

This was caused by the amount of derived variants computed in for Maven

dependencies, which effectively have a very common set of dependencies.

This commit changes that for derived variants and configurations by

de-duplicating MavenDependencyDescriptor instances for serialization.

Issue #8311

    • -2
    • +4
    ./ModuleComponentResolveMetadataSerializer.java
  1. … 3 more files in changeset.
Support requested capabilities on external dependencies

This commit adds support for having requested capabilities

part of the module component selector, for external dependencies.

This means that if a component is using Gradle metadata, we can

read requested capabilities and honor them during selection.

This reworks where requested capabilities are stored, and in

particular moves them to the `ComponentSelector`, making them

properly part of the identity of a dependency. As such, two

dependencies requiring two different variants by using distinct

capabilities will now properly appear as two different dependencies

in the dependency graph, instead of two variants of the same

dependency.

  1. … 63 more files in changeset.
Code style improvements

Support concurrent access in DefaultCachedMetadata

The code was optimized to have a singleton map as much as possible. But

lacked the necessary protections to move out of that case.

As soon as we move beyond a single mapping, we use a concurrent map

underneath and the map swap is now properly protected.

Revert introduction of context object

Since the strategy for isolating caches changed, this is not

necessary anymore.

    • -40
    • +0
    ./DefaultModuleMetadataCacheDetails.java
  1. … 2 more files in changeset.
Revert introduction of context object

Since the strategy for isolating caches changed, this is not

necessary anymore.

    • -40
    • +0
    ./DefaultModuleMetadataCacheDetails.java
  1. … 2 more files in changeset.
Use builders with expected size

This should in theory make things a bit faster.

Use builders with expected size

This should in theory make things a bit faster.

Make comparison of `ModuleComponentAtRepositoryKey` faster

to mitigate small performance regression due to the isolation

of component metadata rules.

  1. … 6 more files in changeset.
Make comparison of `ModuleComponentAtRepositoryKey` faster

to mitigate small performance regression due to the isolation

of component metadata rules.

  1. … 6 more files in changeset.
Use `int` instead of `HashCode`

As `HashCode` was only overhead.

  1. … 4 more files in changeset.
Use `int` instead of `HashCode`

As `HashCode` was only overhead.

  1. … 4 more files in changeset.
Fix isolation of processed metadata

Introduce a hash of the component metadata rules, so that we can

cache different processed metadata based on the hash. This is a

performance optimization, which allows setting different processed

metadata on the "cached metadata" instance, based on a rules hash.

The rule hash is more stable when using class-based rules. If not,

then we have instance based rules which are opaque to Gradle, meaning

that we're going to create as many instances of processed metadata

as there are rules instantiated.

    • -138
    • +0
    ./DefaultModuleMetadataCache.java
    • -0
    • +1
    ./DefaultModuleMetadataCacheDetails.java
    • -0
    • +138
    ./PersistentModuleMetadataCache.java
  1. … 12 more files in changeset.