Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Serialize more details of the project hierachy to the instant execution cache, so that each project's project directory is correct.

Also correctly recreate the project hierarchy. Previously some projects would not be connected to their parent project.

  1. … 18 more files in changeset.
Make sure containers are always shutdown

This commit adds a safety net in case someone forgets to close a container:

a default timeout of 2 minutes would make it happen. If, for some reason,

the code under test takes more time, the timeout can be adjusted.

Add test showing we can keep-alive daemons in containers

  1. … 1 more file in changeset.
Add test case for read-only cache with containers

This commit introduces a test case to check that it's actually

possible to run concurrent docker containers using the same

shared, read-only, dependency cache. It acts both as a test

case and documentation, as mounting the cache as a read-only

volume in Docker is the recommended behavior.

    • -0
    • +84
    ./groovy/org/gradle/containers/GradleContainer.groovy
    • -0
    • +28
    ./groovy/org/gradle/containers/GradleExecResult.groovy
  1. … 15 more files in changeset.
Re-annotate some tests still leaking file handles

Signed-off-by: Paul Merlin <paul@gradle.com>

De-skip tests that don't leak files anymore

but still fail with instant execution

Signed-off-by: Paul Merlin <paul@gradle.com>

  1. … 18 more files in changeset.
Rename @ToBeFixedForInstantExecution.value to skip for explicitness

Signed-off-by: Paul Merlin <paul@gradle.com>

  1. … 36 more files in changeset.
Go over all @ToBeFixedForInstantExecution(Skip.FAILS_IN_SUBCLASS) cases

Signed-off-by: Paul Merlin <paul@gradle.com>

  1. … 14 more files in changeset.
Fix the serialization of `ArtifactCollection` instances that contain the output of artifact transforms to the instant execution cache.

Use a similar strategy to that used to capture the contents of a `FileCollection` that contains the output of artifact transforms.

  1. … 7 more files in changeset.
Do not fail when writing an artifact transform that takes the upstream dependencies of the artifact to the instant execution cache.

In this change, the result will be incorrect because an empty set of dependencies is passed to the transform action when it is loaded from the cache.

  1. … 9 more files in changeset.
Serialize the parameters of an artifact transform to the instant execution cache, rather than attempting to isolate the parameters and then serializing the result.

This allows the parameters to include files and other inputs that may need to be built before they can be queried, for example when the output of some transform is used as an input parameter to another transform (which is something different to chaining of several transforms to produce an output). An implication of this change is that the artifact parameter isolation now happens every time the cache is reused, whereas previously the isolation happened once on write. This can be improved later.

  1. … 11 more files in changeset.
Add API to disable dependency verification

This commit adds an API to disable verification on a specific

configuration (using `resolutionStrategy.disableDependencyVerification`.

This would let tasks which perform special dependency resolution (like

checking newer versions of dependencies) to pass even if dependency

verification is enabled.

  1. … 11 more files in changeset.
Add an XML schema for the verification file

  1. … 4 more files in changeset.
Allow signature verification file generation

This commit adds the ability to generate a verification file which relies

on PGP signature verification. With this mode, Gradle will download the

signatures and verify them. Depending on the result of verification,

Gradle will either:

- automatically add trusted keys if verification passed

- automatically ignore keys if they couldn't be downloaded

- automatically ignore keys if verification failed

If verification failed or that a key couldn't be downloaded, a

WARNING will be issued to encourage the user to verify what

happened.

In order to reduce the size of the verification file, Gradle will

also automatically perform "normalization" of verifications by

configuring globally trusted keys for artifacts which share the same

group or a common super group.

  1. … 22 more files in changeset.
Add ability to ignore keys for a specific artifact

The use case for this is whenever signature for an artifact fails, but

for some reason the user still trusts the artifact. For example, a POM

file is different between different repositories because it happened

to be published twice with different timestamps.

In this case it is recommended to ignore the signature, however we

_will_ fallback on checksum verification.

  1. … 13 more files in changeset.
Add a signature verification cache

This cache avoids re-checking signatures on every build, or even for

the same file multiple times during a build.

  1. … 12 more files in changeset.
Add support for globally trusted keys

A globally trusted key can be used to trust a number of

modules and greatly simplifies configuration: instead of

having to specify checksums for all modules, a user can

declare the keys they trust and use a similar syntax to

trusted artifacts to say to what group/name/version the

key applies.

It's often the case that the same keys are used for

several artifacts of the same group or same company, so

this makes it possible to avoid a lot of boilerplate as

long as the artifacts are signed by the same keys.

  1. … 9 more files in changeset.
Add support for ignored keys

Ignored keys can be used in case verification of a signature isn't

possible because a key isn't available anymore (lost, not published

to a key server, ...).

It's worth noting that if a component cannot be verified by at least

one public key, then verification will fallback to checksum verification.

  1. … 17 more files in changeset.
Make sure we can verify artifacts downloaded in a previous build

  1. … 3 more files in changeset.
Initial implementation of verification of signatures

This commit introduces _signature_ verification. Signature verification

is stronger than checksum verification and must be enabled explicitly,

by adding `<signature-verification>true</signature-verification>` to the

dependency verification configuration file.

Once such verification is enabled, Gradle will do its best to verify

the signature of artifacts. This means:

- it will try to download the .asc file associated with an artifact

- if it's present, it will automatically download the public keys

of the signature and verify that the file matches the signatures

- if _any_ of the signature verification fails, fails the build

- if a public key is not trusted explicitly, fails the build

- if signature verification succeeds, no checksum verification is

performed

Currently it's not possible to perform checksum verification for some

modules and signature verification for others. All modules must declare

all trusted keys.

If a key cannot be downloaded, verification will fail. It's not possible

to ignore a key for now. It's not possible to fallback to checksum

verification.

  1. … 61 more files in changeset.
Allow a build services to be used as the parameter for isolated objects, such as other build services, artifact transforms or worker API actions.

There are still some missing pieces to this:

- Worker classloader and process isolation is not supported.

- Services are stopped in the order they are created, rather than in reverse dependency order.

- Parallel usage constraints specified for these services are not honoured.

  1. … 17 more files in changeset.
Create a different verification file if dry run is on

This commit is another usability issue: this allows simulating

the write of a verification file by resolving all configurations,

but instead of modifying the file it actually generates a different

one which can then be used for diff'ing.

This can be useful whenever the user wants to have comments and a

specific order in their verification file and want to merge the

diff by hand. It's also useful to avoid having to actually execute

the build to check if all configurations would be ok.

  1. … 3 more files in changeset.
Ignore trusted modules when writing the file

This is particularly useful when you want to ignore some modules

in the output file (for example, the native-platform snapshots)

but don't want them to appear in the verification file either.

The checking is also slightly changed so that we would only check

if a module is trusted if there's actually a verification error.

  1. … 4 more files in changeset.
Make it possible to trust some modules

There are cases where it makes sense to trust some modules.

For example, a company using a frequent release pace may want

to trust their company artifacts (changing often so painful

to update the configuration) more than the external dependencies.

This gives the opportunity to tell what are the trusted modules.

The configuration requires at least a group name, but modules

can be trusted on the whole (group, name, version, file name)

tuple.

It is also possible to use regular expressions, for example one

could use:

<trust group="com[.]mycompany[.].*"/>

  1. … 10 more files in changeset.
Make it possible to disable metadata verification

This commit introduces basic configuration for dependency

verification. The only thing that is configurable now is

the ability to disable verification of metadata. This can

be useful whenever the user only wants to trust artifacts,

because addition of metadata in verification files can

be quite verbose.

  1. … 14 more files in changeset.
Make verification model more resilient to real world projects

Dogfooding the Gradle build with dependenvy verification proved to

be helpful. There are quite a few cases where we discover dependencies

which come from different repositories. Reposiories can also be mirrored

and sometimes the mirror doesn't mirror what is was supposed to.

The problem is that working around, for example by fixing the mirrors

or figuring out how to fetch a dependency from the right place can be

tricky. It's often easier to go and check the dependency and/or metadata

and approve it.

For this purpose, the verification metadata file now includes the

ability to have "alternate", trusted checksums. It also adds the ability

to tell where a checksum comes from, as indication to the reader. Checksums

generated by Gradle will be marked as such, and therefore a reader can

see that they are less "trustworthy" than checksums fetched by a human.

  1. … 12 more files in changeset.
De-duplicate entries based on file name instead of artifact id

Because Gradle internally sometimes uses `DefaultModuleComponentArtifactIdentifier`

or `ModuleComponentFileArtifactIdentifier` for the same artifact depending on the

context, we can't rely on equality here. This commit changes the internal verification

structure to rely on the file name which is more consistent and fixes duplication

issues.

  1. … 8 more files in changeset.
Use file names instea of Ivy artifact names for comparison

This commit reworks the generation of verification file and

verification itself in order to use the file name instead of

the Ivy artifact name. This is done because in case of Gradle

module metadata, the file name of an artifact is not necessarily

directly bound to the module name and causes comparison issues.

  1. … 10 more files in changeset.
Introduce module metadata verification

This commit introduces verification of metadata files.

For this, another refactoring of the `ModuleSource` concept

was required. Ironically, before this commit, `ModuleSource`

used to be available for serialization in the module metadata

serializer. However, they weren't used in practice, because

all the required data could be reconstructed from the caches.

In particular, there was this "contentHash" which, because

not properly serialized, was actually set as a field on the

component resolve metadata itself, instead of being part

of the module source.

Now, this commit reintroduces serialization of module sources

but takes a different approach by splitting the module sources

in two distinct categories:

- module sources which can be reconstructed from known data,

such as the repository name and repository url

- module sources which have to be serialized alongside component

metadata, because they can't be reconstructed from sources

The latter category includes this "contentHash", serialized with

the descriptor hash module source. It also includes the information

about _which_ actual descriptor file was used to generate the

binary module descriptor (e.g, the source POM, Ivy or module

metadata file). This information does _not_ belong to the module

component resolve metadata itself, so it belongs to its sources.

For this purpose, serialization of module sources has been updated

so that instead of using Java serialization, module sources need

to provide a custom serializer, called a Codec. Those codecs are

uniquely identified by an id which is just an integer. This is

done for performance optimization, in order to avoid to serialize

the name of the codec and have to load it dynamically. Instead,

Gradle knows about the full set of serializers (and there's no

way for a user to add more because in any case it would require

an update of the module metadata store format).

This makes it much more efficient to serialize module sources

(because we can now have an optimized encoder), but it also

permits reconstructing module sources from incomplete information.

In particular, the module source which describes the file from

which a component resolve metadata was sourced contains a link

to the actual file in the local artifact store. However, in order

to be relocatable, we _don't_ want this file path to be stored

in the metadata cache. This means that instead of storing the

path, we actually store the artifact identifier and the hash

of the descriptor so that we can, when loaded from cache, find

its location back.

Currently, metadata verification is enabled for all components.

It's not possible to disable verification of metadata.

  1. … 39 more files in changeset.
Add standard setup for a published platform (GMM or BOM) to test fixture