Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Make it mandatory to choose checksum type

Otherwise when updating the file you may accidentally generate checksums

you didn't intend to.

    • -1
    • +1
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 3 more files in changeset.
Make error message more actionable

    • -2
    • +2
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 1 more file in changeset.
Create a different verification file if dry run is on

This commit is another usability issue: this allows simulating

the write of a verification file by resolving all configurations,

but instead of modifying the file it actually generates a different

one which can then be used for diff'ing.

This can be useful whenever the user wants to have comments and a

specific order in their verification file and want to merge the

diff by hand. It's also useful to avoid having to actually execute

the build to check if all configurations would be ok.

    • -0
    • +103
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 3 more files in changeset.
Ignore trusted modules when writing the file

This is particularly useful when you want to ignore some modules

in the output file (for example, the native-platform snapshots)

but don't want them to appear in the verification file either.

The checking is also slightly changed so that we would only check

if a module is trusted if there's actually a verification error.

    • -0
    • +83
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 4 more files in changeset.
Improve error handling when the verification file cannot be read

Before this change, a failure to read the file would prevent the

services from being initialized, which wasn't user friendly. Now

the error is deferred when the actual resolution happens.

    • -0
    • +21
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 1 more file in changeset.
Mark test as to be fixed for instant execution

    • -0
    • +1
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
Fix merge conflict

    • -2
    • +10
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 2 more files in changeset.
Poll for work in progress line in test

instead of using sleep.

    • -11
    • +24
    ./resolve/transform/TransformationLoggingIntegrationTest.groovy
Add lenient version of ComponentMetadataDetails.addVariant() (#11565)

    • -10
    • +90
    ./resolve/rules/VariantFilesMetadataRulesIntegrationTest.groovy
  1. … 10 more files in changeset.
Test that transforms print > Transforming

since those messages will be filtered from the build scan logs.

    • -0
    • +50
    ./resolve/transform/TransformationLoggingIntegrationTest.groovy
Make it possible to trust some modules

There are cases where it makes sense to trust some modules.

For example, a company using a frequent release pace may want

to trust their company artifacts (changing often so painful

to update the configuration) more than the external dependencies.

This gives the opportunity to tell what are the trusted modules.

The configuration requires at least a group name, but modules

can be trusted on the whole (group, name, version, file name)

tuple.

It is also possible to use regular expressions, for example one

could use:

<trust group="com[.]mycompany[.].*"/>

    • -0
    • +30
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -2
    • +9
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 9 more files in changeset.
Make it possible to disable metadata verification

This commit introduces basic configuration for dependency

verification. The only thing that is configurable now is

the ability to disable verification of metadata. This can

be useful whenever the user only wants to trust artifacts,

because addition of metadata in verification files can

be quite verbose.

    • -1
    • +49
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -0
    • +107
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 13 more files in changeset.
Fix writing/verification of alternate checksums

    • -0
    • +33
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -1
    • +60
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 2 more files in changeset.
Make verification model more resilient to real world projects

Dogfooding the Gradle build with dependenvy verification proved to

be helpful. There are quite a few cases where we discover dependencies

which come from different repositories. Reposiories can also be mirrored

and sometimes the mirror doesn't mirror what is was supposed to.

The problem is that working around, for example by fixing the mirrors

or figuring out how to fetch a dependency from the right place can be

tricky. It's often easier to go and check the dependency and/or metadata

and approve it.

For this purpose, the verification metadata file now includes the

ability to have "alternate", trusted checksums. It also adds the ability

to tell where a checksum comes from, as indication to the reader. Checksums

generated by Gradle will be marked as such, and therefore a reader can

see that they are less "trustworthy" than checksums fetched by a human.

    • -24
    • +23
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 12 more files in changeset.
De-duplicate entries based on file name instead of artifact id

Because Gradle internally sometimes uses `DefaultModuleComponentArtifactIdentifier`

or `ModuleComponentFileArtifactIdentifier` for the same artifact depending on the

context, we can't rely on equality here. This commit changes the internal verification

structure to rely on the file name which is more consistent and fixes duplication

issues.

    • -0
    • +81
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 8 more files in changeset.
Restore accidentally deleted code

    • -13
    • +13
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -22
    • +22
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 1 more file in changeset.
Sort conflict participants

In some corner cases, it matters to select first the conflict winner

before attempting any other selection.

Fixes #11569

    • -0
    • +10
    ./resolve/rules/ComponentReplacementIntegrationTest.groovy
  1. … 1 more file in changeset.
Introduce a checksum file cache service

This service is responsible for caching the checksums computed from

local file system. Because it's also used for dependency verification

writing and checking, this cache uses the existing infrastructure which

makes sure that if a file is updated locally, we expire the entry in

the cache.

This is done because there are lots of places in the code where we

used the legacy `HashUtil` class, which has no caching whatsoever.

It's, however, quite common to have a build which generates sha1

checksums multiple times for the same file. For example, during

publication.

    • -6
    • +9
    ./resolve/caching/CachedDependencyResolutionIntegrationTest.groovy
    • -3
    • +3
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 101 more files in changeset.
Always serialize module sources

Module sources were only serialized in the cache metadata entry.

In practice, they belong to the module metadata, so they are now

properly serialized as part of it. This fixes the "force realize"

tests.

    • -7
    • +8
    ./resolve/rules/ComponentMetadataRulesCachingIntegrationTest.groovy
    • -1
    • +2
    ./resolve/verification/AbstractDependencyVerificationIntegTest.groovy
  1. … 17 more files in changeset.
Add more test coverage for deleted artifacts

    • -0
    • +100
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 5 more files in changeset.
Track composition of Maven module metadata

The binary representation of a Maven POM was only tracking the

POM file itself as the source. However, a POM can reference

parent POMs which also participate in resolution. This commit

changes the `ModuleSources` so that we can query more than one

module source of a specific type, allowing us to store more

than a single module source for a single file.

    • -3
    • +2
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 4 more files in changeset.
Bump layout format

    • -9
    • +9
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -22
    • +22
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 5 more files in changeset.
Use file names instea of Ivy artifact names for comparison

This commit reworks the generation of verification file and

verification itself in order to use the file name instead of

the Ivy artifact name. This is done because in case of Gradle

module metadata, the file name of an artifact is not necessarily

directly bound to the module name and causes comparison issues.

    • -32
    • +35
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 10 more files in changeset.
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

    • -6
    • +16
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -3
    • +189
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 38 more files in changeset.
Ignore changing modules for dependency verification

    • -0
    • +25
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
    • -0
    • +23
    ./resolve/verification/DependencyVerificationWritingIntegTest.groovy
  1. … 1 more file in changeset.
Improve error for non resolvable configurations

When attempting to resolve a configuration that is canBeResolved=false,

the error now mentions it is a dependency configuration and that

resolution "on its own" is invalid.

    • -3
    • +3
    ./resolve/api/ConfigurationRoleIntegrationTest.groovy
  1. … 3 more files in changeset.
Add dependency verification mode

This commit introduces a selection of the dependency verification

mode. There are 3 different modes: strict (default), lenient and

off.

The lenient mode has been added because dogfooding the feature showed

it could be painful to udpate the dependency verification

metadata file without such an option: often we want to diagnose

why a dependency is here, but as soon as the dependency

verification file is present, verification is active and immediately

fails the build.

This effectively prevents from using the usual tooling to check

where a dependency comes from. The workaround was to temporarily

rename the file, run the build, then rename again, which was tedious.

So, instead of adding a mode where verification would be totally

ignored, this commit introduces a mode where the errors are turned

into warnings. This doesn't totally silence the problems, which

makes them more visible to the developer.

The "off" mode is for use cases where users prefer not to deal

with upgrades locally, but want to see verification happening

on CI only.

    • -0
    • +93
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy
  1. … 8 more files in changeset.
Improve message for invalid configuration mutation

This now indicates what is being modified is a "dependency

configuration". Also what is being modified is now more explicit in some

cases.

    • -4
    • +4
    ./resolve/OutgoingVariantsMutationIntegrationTest.groovy
    • -2
    • +2
    ./resolve/ProjectDependencyResolveIntegrationTest.groovy
    • -1
    • +1
    ./resolve/api/ConfigurationDefaultsIntegrationTest.groovy
    • -2
    • +2
    ./resolve/api/ConfigurationMutationIntegrationTest.groovy
    • -22
    • +22
    ./resolve/api/UnsupportedConfigurationMutationTest.groovy
  1. … 5 more files in changeset.
Add missing test exclusions for instant execution

    • -0
    • +4
    ./resolve/locking/AbstractLockingIntegrationTest.groovy
    • -0
    • +3
    ./resolve/locking/DependencyLockingLenientModeIntegrationTest.groovy
    • -0
    • +4
    ./resolve/locking/DependencyLockingStrictModeIntegrationTest.groovy
Remove @ToBeFixedForInstantExecution for test that seems to be passing

    • -1
    • +0
    ./resolve/verification/DependencyVerificationIntegrityCheckIntegTest.groovy