Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

    • -6
    • +16
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -3
    • +189
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 38 more files in changeset.
Ignore changing modules for dependency verification

    • -0
    • +25
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -0
    • +23
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 1 more file in changeset.
Ignore changing modules for dependency verification

    • -0
    • +25
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -0
    • +23
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 1 more file in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Add tests for rewiring compile task dependencies

This also updates existing tests to use the new API.

    • -1
    • +1
    ./resolve/attributes/VariantAwareResolutionWithConfigurationAttributesIntegrationTest.groovy
  1. … 15 more files in changeset.
Improve error for non resolvable configurations

When attempting to resolve a configuration that is canBeResolved=false,

the error now mentions it is a dependency configuration and that

resolution "on its own" is invalid.

    • -3
    • +3
    ./resolve/api/ConfigurationRoleIntegrationTest.groovy
  1. … 3 more files in changeset.
Improve error for non resolvable configurations

When attempting to resolve a configuration that is canBeResolved=false,

the error now mentions it is a dependency configuration and that

resolution "on its own" is invalid.

    • -3
    • +3
    ./resolve/api/ConfigurationRoleIntegrationTest.groovy
  1. … 3 more files in changeset.
Improve error for non resolvable configurations

When attempting to resolve a configuration that is canBeResolved=false,

the error now mentions it is a dependency configuration and that

resolution "on its own" is invalid.

    • -3
    • +3
    ./resolve/api/ConfigurationRoleIntegrationTest.groovy
  1. … 3 more files in changeset.
Improve error for non resolvable configurations

When attempting to resolve a configuration that is canBeResolved=false,

the error now mentions it is a dependency configuration and that

resolution "on its own" is invalid.

    • -3
    • +3
    ./resolve/api/ConfigurationRoleIntegrationTest.groovy
  1. … 3 more files in changeset.
Improve error for non resolvable configurations

When attempting to resolve a configuration that is canBeResolved=false,

the error now mentions it is a dependency configuration and that

resolution "on its own" is invalid.

    • -3
    • +3
    ./resolve/api/ConfigurationRoleIntegrationTest.groovy
  1. … 3 more files in changeset.
Improve error for non resolvable configurations

When attempting to resolve a configuration that is canBeResolved=false,

the error now mentions it is a dependency configuration and that

resolution "on its own" is invalid.

    • -3
    • +3
    ./resolve/api/ConfigurationRoleIntegrationTest.groovy
  1. … 3 more files in changeset.
Add dependency verification mode

This commit introduces a selection of the dependency verification

mode. There are 3 different modes: strict (default), lenient and

off.

The lenient mode has been added because dogfooding the feature showed

it could be painful to udpate the dependency verification

metadata file without such an option: often we want to diagnose

why a dependency is here, but as soon as the dependency

verification file is present, verification is active and immediately

fails the build.

This effectively prevents from using the usual tooling to check

where a dependency comes from. The workaround was to temporarily

rename the file, run the build, then rename again, which was tedious.

So, instead of adding a mode where verification would be totally

ignored, this commit introduces a mode where the errors are turned

into warnings. This doesn't totally silence the problems, which

makes them more visible to the developer.

The "off" mode is for use cases where users prefer not to deal

with upgrades locally, but want to see verification happening

on CI only.

    • -0
    • +93
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 8 more files in changeset.
Add lenient dependency verification mode

This mode has been added because dogfooding the feature showed

it could be painful to udpate the dependency verification

metadata file without such an option: often we want to diagnose

why a dependency is here, but as soon as the dependency

verification file is present, verification is active and immediately

fails the build.

This effectively prevents from using the usual tooling to check

where a dependency comes from. The workaround was to temporarily

rename the file, run the build, then rename again, which was tedious.

So, instead of adding a mode where verification would be totally

ignored, this commit introduces a mode where the errors are turned

into warnings. This doesn't totally silence the problems, which

makes them more visible to the developer.

    • -0
    • +25
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 5 more files in changeset.
Add dependency verification mode

This commit introduces a selection of the dependency verification

mode. There are 3 different modes: strict (default), lenient and

off.

The lenient mode has been added because dogfooding the feature showed

it could be painful to udpate the dependency verification

metadata file without such an option: often we want to diagnose

why a dependency is here, but as soon as the dependency

verification file is present, verification is active and immediately

fails the build.

This effectively prevents from using the usual tooling to check

where a dependency comes from. The workaround was to temporarily

rename the file, run the build, then rename again, which was tedious.

So, instead of adding a mode where verification would be totally

ignored, this commit introduces a mode where the errors are turned

into warnings. This doesn't totally silence the problems, which

makes them more visible to the developer.

The "off" mode is for use cases where users prefer not to deal

with upgrades locally, but want to see verification happening

on CI only.

    • -0
    • +93
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 8 more files in changeset.
Add lenient dependency verification mode

This mode has been added because dogfooding the feature showed

it could be painful to udpate the dependency verification

metadata file without such an option: often we want to diagnose

why a dependency is here, but as soon as the dependency

verification file is present, verification is active and immediately

fails the build.

This effectively prevents from using the usual tooling to check

where a dependency comes from. The workaround was to temporarily

rename the file, run the build, then rename again, which was tedious.

So, instead of adding a mode where verification would be totally

ignored, this commit introduces a mode where the errors are turned

into warnings. This doesn't totally silence the problems, which

makes them more visible to the developer.

    • -0
    • +25
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 5 more files in changeset.
Document current behavior of writing parent POM file verification

The current implementation of dependency verification metadata

generation computes POM metadata verification checksums for parent

POMs, _only when they are downloaded during the same build_.

This behavior is an artifact of how parent POM resolution is

implemented. Eventually, all metadata should be considered equal

and have metadata written in the verification file.

    • -0
    • +78
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 1 more file in changeset.
Improve message for invalid configuration mutation

This now indicates what is being modified is a "dependency

configuration". Also what is being modified is now more explicit in some

cases.

    • -4
    • +4
    ./resolve/OutgoingVariantsMutationIntegrationTest.groovy
    • -2
    • +2
    ./resolve/ProjectDependencyResolveIntegrationTest.groovy
    • -1
    • +1
    ./resolve/api/ConfigurationDefaultsIntegrationTest.groovy
    • -2
    • +2
    ./resolve/api/ConfigurationMutationIntegrationTest.groovy
    • -22
    • +22
    ./resolve/api/UnsupportedConfigurationMutationTest.groovy
  1. … 5 more files in changeset.
Improve message for invalid configuration mutation

This now indicates what is being modified is a "dependency

configuration". Also what is being modified is now more explicit in some

cases.

    • -4
    • +4
    ./resolve/OutgoingVariantsMutationIntegrationTest.groovy
    • -2
    • +2
    ./resolve/ProjectDependencyResolveIntegrationTest.groovy
    • -1
    • +1
    ./resolve/api/ConfigurationDefaultsIntegrationTest.groovy
    • -2
    • +2
    ./resolve/api/ConfigurationMutationIntegrationTest.groovy
    • -22
    • +22
    ./resolve/api/UnsupportedConfigurationMutationTest.groovy
  1. … 5 more files in changeset.
Improve message for invalid configuration mutation

This now indicates what is being modified is a "dependency

configuration". Also what is being modified is now more explicit in some

cases.

    • -4
    • +4
    ./resolve/OutgoingVariantsMutationIntegrationTest.groovy
    • -2
    • +2
    ./resolve/ProjectDependencyResolveIntegrationTest.groovy
    • -1
    • +1
    ./resolve/api/ConfigurationDefaultsIntegrationTest.groovy
    • -2
    • +2
    ./resolve/api/ConfigurationMutationIntegrationTest.groovy
    • -22
    • +22
    ./resolve/api/UnsupportedConfigurationMutationTest.groovy
  1. … 5 more files in changeset.
Improve message for invalid configuration mutation

This now indicates what is being modified is a "dependency

configuration". Also what is being modified is now more explicit in some

cases.

    • -4
    • +4
    ./resolve/OutgoingVariantsMutationIntegrationTest.groovy
    • -2
    • +2
    ./resolve/ProjectDependencyResolveIntegrationTest.groovy
    • -1
    • +1
    ./resolve/api/ConfigurationDefaultsIntegrationTest.groovy
    • -2
    • +2
    ./resolve/api/ConfigurationMutationIntegrationTest.groovy
    • -22
    • +22
    ./resolve/api/UnsupportedConfigurationMutationTest.groovy
  1. … 5 more files in changeset.
Improve message for invalid configuration mutation

This now indicates what is being modified is a "dependency

configuration". Also what is being modified is now more explicit in some

cases.

    • -4
    • +4
    ./resolve/OutgoingVariantsMutationIntegrationTest.groovy
    • -2
    • +2
    ./resolve/ProjectDependencyResolveIntegrationTest.groovy
    • -1
    • +1
    ./resolve/api/ConfigurationDefaultsIntegrationTest.groovy
    • -2
    • +2
    ./resolve/api/ConfigurationMutationIntegrationTest.groovy
    • -22
    • +22
    ./resolve/api/UnsupportedConfigurationMutationTest.groovy
  1. … 5 more files in changeset.
Improve message for invalid configuration mutation

This now indicates what is being modified is a "dependency

configuration". Also what is being modified is now more explicit in some

cases.

    • -4
    • +4
    ./resolve/OutgoingVariantsMutationIntegrationTest.groovy
    • -2
    • +2
    ./resolve/ProjectDependencyResolveIntegrationTest.groovy
    • -1
    • +1
    ./resolve/api/ConfigurationDefaultsIntegrationTest.groovy
    • -2
    • +2
    ./resolve/api/ConfigurationMutationIntegrationTest.groovy
    • -22
    • +22
    ./resolve/api/UnsupportedConfigurationMutationTest.groovy
  1. … 5 more files in changeset.
Add missing test exclusions for instant execution

    • -0
    • +4
    ./resolve/locking/AbstractLockingIntegrationTest.groovy
    • -0
    • +3
    ./resolve/locking/DependencyLockingLenientModeIntegrationTest.groovy
    • -0
    • +4
    ./resolve/locking/DependencyLockingStrictModeIntegrationTest.groovy
Add missing test exclusions for instant execution

    • -0
    • +4
    ./resolve/locking/AbstractLockingIntegrationTest.groovy
    • -0
    • +3
    ./resolve/locking/DependencyLockingLenientModeIntegrationTest.groovy
    • -0
    • +4
    ./resolve/locking/DependencyLockingStrictModeIntegrationTest.groovy