Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Consider the derivation strategy when requesting component metadata

Gradle has a single, build scoped cache for dependency metadata. It's

using a build cache scope because component metadata is quite expensive

to read and process. However, component metadata _may_ be different

based on the plugins applied on a project. For example, Java projects

apply what we call a "Java derivation strategy", which is capable

of transforming regular POM files into so-called "platforms" but

more importantly, they are what map the different Maven scopes to

actual "variants" in variant-aware dependency management.

The problem is that it's not a requirement that all projects are

Java projects. It is possible to have native subprojects, or projects

which do _not_ apply the Java base plugin. While it's in general an

error to have a project which wants to perform "Java dependency

resolution" not to apply the Java derivation strategy, there are

consequences in not doing so. In particular, Gradle becomes order

dependent: if a subproject which does not apply the derivation

strategy resolves a module first, then subsequent resolutions for

projects which _do_ apply the derivation strategy would get wrong

metadata.

To avoid this, this commit makes sure that when we consume module

metadata, we actually use the derivation strategy as part of the

processed metadata cache key.

This does _not_ fix the fact that the consumers _should have_ applied

the derivation strategy and will make realizing this harder, but it

_does_ fix the inconsistency in resolution, which is probably a

much bigger issue (for caching and reproducibility).

    • -1
    • +1
    ./model/DefaultMavenModuleResolveMetadataTest.groovy
    • -1
    • +1
    ./model/DependencyConstraintMetadataRulesTest.groovy
    • -1
    • +1
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 10 more files in changeset.
Consider the derivation strategy when requesting component metadata

Gradle has a single, build scoped cache for dependency metadata. It's

using a build cache scope because component metadata is quite expensive

to read and process. However, component metadata _may_ be different

based on the plugins applied on a project. For example, Java projects

apply what we call a "Java derivation strategy", which is capable

of transforming regular POM files into so-called "platforms" but

more importantly, they are what map the different Maven scopes to

actual "variants" in variant-aware dependency management.

The problem is that it's not a requirement that all projects are

Java projects. It is possible to have native subprojects, or projects

which do _not_ apply the Java base plugin. While it's in general an

error to have a project which wants to perform "Java dependency

resolution" not to apply the Java derivation strategy, there are

consequences in not doing so. In particular, Gradle becomes order

dependent: if a subproject which does not apply the derivation

strategy resolves a module first, then subsequent resolutions for

projects which _do_ apply the derivation strategy would get wrong

metadata.

To avoid this, this commit makes sure that when we consume module

metadata, we actually use the derivation strategy as part of the

processed metadata cache key.

This does _not_ fix the fact that the consumers _should have_ applied

the derivation strategy and will make realizing this harder, but it

_does_ fix the inconsistency in resolution, which is probably a

much bigger issue (for caching and reproducibility).

    • -1
    • +1
    ./model/DefaultMavenModuleResolveMetadataTest.groovy
    • -1
    • +1
    ./model/DependencyConstraintMetadataRulesTest.groovy
    • -1
    • +1
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 10 more files in changeset.
Force AbstractTestDirectoryProvider to use Class (#12431)

Closes https://github.com/gradle/gradle-private/issues/2988

This PR adds `className` to `AbstractTestDirectoryProvider` so there'll be no more `unknown-test-class`.

    • -1
    • +1
    ./ivypublish/DefaultIvyModuleDescriptorWriterTest.groovy
  1. … 407 more files in changeset.
Fix more construtors

    • -1
    • +1
    ./ivypublish/DefaultIvyModuleDescriptorWriterTest.groovy
  1. … 3 more files in changeset.
Revert some changes to artifact transform execution, as these changes introduce a performance regression.

    • -4
    • +4
    ./model/DefaultModuleComponentArtifactIdentifierTest.groovy
  1. … 30 more files in changeset.
Revert some changes to artifact transform execution, as these changes introduce a performance regression.

    • -4
    • +4
    ./model/DefaultModuleComponentArtifactIdentifierTest.groovy
  1. … 30 more files in changeset.
Fixes for previous commit.

    • -4
    • +4
    ./model/DefaultModuleComponentArtifactIdentifierTest.groovy
  1. … 3 more files in changeset.
Fixes for previous commit.

    • -4
    • +4
    ./model/DefaultModuleComponentArtifactIdentifierTest.groovy
  1. … 3 more files in changeset.
Merge some logic used for executing a chained scheduled artifact transform node and the other places artifact transforms are executed.

An implication of this change is that when a scheduled transform produces multiple output files, then a consuming scheduled transform will transform those output files in parallel.

    • -4
    • +4
    ./model/DefaultModuleComponentArtifactIdentifierTest.groovy
  1. … 23 more files in changeset.
Merge some logic used for executing a chained scheduled artifact transform node and the other places artifact transforms are executed.

An implication of this change is that when a scheduled transform produces multiple output files, then a consuming scheduled transform will transform those output files in parallel.

    • -4
    • +4
    ./model/DefaultModuleComponentArtifactIdentifierTest.groovy
  1. … 23 more files in changeset.
Add lenient version of ComponentMetadataDetails.addVariant() (#11565)

    • -3
    • +27
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 10 more files in changeset.
Add lenient version of ComponentMetadataDetails.addVariant()

    • -3
    • +27
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 10 more files in changeset.
Add lenient version of ComponentMetadataDetails.addVariant()

    • -3
    • +3
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 9 more files in changeset.
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

    • -16
    • +2
    ./model/AbstractMutableModuleComponentResolveMetadataTest.groovy
    • -7
    • +0
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -6
    • +0
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 37 more files in changeset.
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

    • -16
    • +2
    ./model/AbstractMutableModuleComponentResolveMetadataTest.groovy
    • -7
    • +0
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -6
    • +0
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 36 more files in changeset.
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

    • -16
    • +2
    ./model/AbstractMutableModuleComponentResolveMetadataTest.groovy
    • -7
    • +0
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -6
    • +0
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 37 more files in changeset.
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

    • -16
    • +2
    ./model/AbstractMutableModuleComponentResolveMetadataTest.groovy
    • -7
    • +0
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -6
    • +0
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 37 more files in changeset.
Refactor `ModuleSource`

The `ModuleSource` concept was a bit messy. It was designed in order

to be able to store the origin of an artifact. Over time, it evolved

into storing more information, like snapshot timestamps, repositories

or content hash.

The code was convoluted because each part of the code was expecting

some kind of module source, but because of delegation, it wasn't

really possible to add/mix more sources.

This commit refactors this concept into a `ModuleSources` concept

which allows storing more information about a module source, in

a safe and consistent manner. No more wrapping/unwrapping, and each

code requiring a specific type of module source can query for it.

    • -20
    • +0
    ./model/AbstractLazyModuleComponentResolveMetadataTest.groovy
    • -21
    • +0
    ./model/DefaultMavenModuleResolveMetadataTest.groovy
    • -19
    • +21
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -19
    • +21
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 61 more files in changeset.
Refactor `ModuleSource`

The `ModuleSource` concept was a bit messy. It was designed in order

to be able to store the origin of an artifact. Over time, it evolved

into storing more information, like snapshot timestamps, repositories

or content hash.

The code was convoluted because each part of the code was expecting

some kind of module source, but because of delegation, it wasn't

really possible to add/mix more sources.

This commit refactors this concept into a `ModuleSources` concept

which allows storing more information about a module source, in

a safe and consistent manner. No more wrapping/unwrapping, and each

code requiring a specific type of module source can query for it.

    • -20
    • +0
    ./model/AbstractLazyModuleComponentResolveMetadataTest.groovy
    • -21
    • +0
    ./model/DefaultMavenModuleResolveMetadataTest.groovy
    • -19
    • +20
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -19
    • +20
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 59 more files in changeset.
Refactor `ModuleSource`

The `ModuleSource` concept was a bit messy. It was designed in order

to be able to store the origin of an artifact. Over time, it evolved

into storing more information, like snapshot timestamps, repositories

or content hash.

The code was convoluted because each part of the code was expecting

some kind of module source, but because of delegation, it wasn't

really possible to add/mix more sources.

This commit refactors this concept into a `ModuleSources` concept

which allows storing more information about a module source, in

a safe and consistent manner. No more wrapping/unwrapping, and each

code requiring a specific type of module source can query for it.

    • -20
    • +0
    ./model/AbstractLazyModuleComponentResolveMetadataTest.groovy
    • -21
    • +0
    ./model/DefaultMavenModuleResolveMetadataTest.groovy
    • -19
    • +21
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -19
    • +21
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 61 more files in changeset.
Refactor `ModuleSource`

The `ModuleSource` concept was a bit messy. It was designed in order

to be able to store the origin of an artifact. Over time, it evolved

into storing more information, like snapshot timestamps, repositories

or content hash.

The code was convoluted because each part of the code was expecting

some kind of module source, but because of delegation, it wasn't

really possible to add/mix more sources.

This commit refactors this concept into a `ModuleSources` concept

which allows storing more information about a module source, in

a safe and consistent manner. No more wrapping/unwrapping, and each

code requiring a specific type of module source can query for it.

    • -20
    • +0
    ./model/AbstractLazyModuleComponentResolveMetadataTest.groovy
    • -21
    • +0
    ./model/DefaultMavenModuleResolveMetadataTest.groovy
    • -19
    • +21
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -19
    • +21
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 61 more files in changeset.
Refactor `ModuleSource`

The `ModuleSource` concept was a bit messy. It was designed in order

to be able to store the origin of an artifact. Over time, it evolved

into storing more information, like snapshot timestamps, repositories

or content hash.

The code was convoluted because each part of the code was expecting

some kind of module source, but because of delegation, it wasn't

really possible to add/mix more sources.

This commit refactors this concept into a `ModuleSources` concept

which allows storing more information about a module source, in

a safe and consistent manner. No more wrapping/unwrapping, and each

code requiring a specific type of module source can query for it.

    • -20
    • +0
    ./model/AbstractLazyModuleComponentResolveMetadataTest.groovy
    • -21
    • +0
    ./model/DefaultMavenModuleResolveMetadataTest.groovy
    • -19
    • +20
    ./model/DefaultMutableIvyModuleResolveMetadataTest.groovy
    • -19
    • +20
    ./model/DefaultMutableMavenModuleResolveMetadataTest.groovy
  1. … 60 more files in changeset.
Remove `platform` dsl from constraint handler

These shortcuts define details of a dependency like attributes,

requested capabilities and 'endorse strict' status. These things

can not be defined on constraints. So these methods only cause

inconsistent behavior.

One can use constraints in combination with platforms like this to

control platform versions:

dependencies {

api platform("org:platform")

constraints {

api "org:platform:1.0"

}

}

    • -1
    • +0
    ./model/AbstractDependencyMetadataRulesTest.groovy
  1. … 11 more files in changeset.
Remove `platform` dsl from constraint handler

These shortcuts define details of a dependency like attributes,

requested capabilities and 'endorse strict' status. These things

can not be defined on constraints. So these methods only cause

inconsistent behavior.

One can use constraints in combination with platforms like this to

control platform versions:

dependencies {

api platform("org:platform")

constraints {

api "org:platform:1.0"

}

}

    • -1
    • +0
    ./model/AbstractDependencyMetadataRulesTest.groovy
  1. … 9 more files in changeset.
Remove `platform` dsl from constraint handler

These shortcuts define details of a dependency like attributes,

requested capabilities and 'endorse strict' status. These things

can not be defined on constraints. So these methods only cause

inconsistent behavior.

One can use constraints in combination with platforms like this to

control platform versions:

dependencies {

api platform("org:platform")

constraints {

api "org:platform:1.0"

}

}

    • -1
    • +0
    ./model/AbstractDependencyMetadataRulesTest.groovy
  1. … 12 more files in changeset.
Remove `platform` dsl from constraint handler

These shortcuts define details of a dependency like attributes,

requested capabilities and 'endorse strict' status. These things

can not be defined on constraints. So these methods only cause

inconsistent behavior.

One can use constraints in combination with platforms like this to

control platform versions:

dependencies {

api platform("org:platform")

constraints {

api "org:platform:1.0"

}

}

    • -1
    • +0
    ./model/AbstractDependencyMetadataRulesTest.groovy
  1. … 9 more files in changeset.
Fix NPE and tests

    • -0
    • +2
    ./model/AbstractDependencyMetadataRulesTest.groovy
    • -0
    • +1
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 5 more files in changeset.
Fix NPE and tests

    • -0
    • +2
    ./model/AbstractDependencyMetadataRulesTest.groovy
    • -0
    • +1
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 4 more files in changeset.
Use a different 'shadow capability' for enforced platform

This allows both 'regular' and 'enforced' platform of the same

component to be selected. In order to make this work, support

for projects having shadow capability needed to be added.

    • -0
    • +2
    ./model/AbstractDependencyMetadataRulesTest.groovy
    • -0
    • +1
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 11 more files in changeset.
Always apply all Category disambiguation rules

Before, parts of the platform disambiguation were only done when

using the 'java-platform' plugin. However, other consumers

may also require all rules (see #11091)

    • -1
    • +0
    ./model/AbstractDependencyMetadataRulesTest.groovy
    • -1
    • +0
    ./model/VariantFilesMetadataRulesTest.groovy
  1. … 5 more files in changeset.