Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Got rid of separate in-memory caching layer for local repos

Local file repositories now use the regular `CachingModuleComponentRepository`,

but provide in-memory-only versions of the dependency resolution caches

in order to retain the current 'always-refresh' behaviour.

    • -146
    • +0
    ./InMemoryCachedModuleComponentRepository.java
    • -86
    • +0
    ./InMemoryCachedRepositoryFactory.java
    • -42
    • +0
    ./InMemoryModuleComponentRepositoryCaches.java
  1. … 10 more files in changeset.
Remove old feature toggle for in-memory metadata caching

    • -10
    • +0
    ./InMemoryCachedRepositoryFactory.java
  1. … 1 more file in changeset.
Provide in-memory caching for dependency caches

Instead of using a separate `InMemoryCachedModuleComponentRepository`

for in-memory caching of resolution from remote repositories, the

filesystem-backed cache implementations now each have a build-scoped

in-memory cache as well.

This provides more consistent caching behaviour, since any cache-expiry

logic is shared between in-memory and filesystem caches.

For file-backed repositories, we don't use the filesystem-backed cache,

so we continue to use `InMemoryCachedModuleComponentRepository` for

in-memory caching of resolved modules.

Note that this is only a partial fix, as it does not address the

incorrect caching of maven snapshot modules. This is due to the id-based

caching of resolved artifacts, and the fact that different maven

snapshot instances do not have a different unique identifier.

    • -2
    • +2
    ./InMemoryCachedModuleComponentRepository.java
    • -2
    • +41
    ./InMemoryCachedRepositoryFactory.java
  1. … 7 more files in changeset.
Simple fix for cache-expiry by removing in-memory cache

This fix demonstrates that the incorrect dynamic version behaviour

of gradle#3019 is due to the in-memory cache of module version lists.

A more complete fix will involve moving the in-memory caching to live

with the other filesystem caching, allowing the cache expiry policy

to be correctly honoured.

    • -2
    • +2
    ./InMemoryCachedModuleComponentRepository.java
  1. … 3 more files in changeset.
Javadoc for `ModuleComponentReposotory` types

    • -0
    • +7
    ./InMemoryCachedModuleComponentRepository.java
  1. … 5 more files in changeset.
Convert many uses of ModuleVersionSelector -> ModuleComponentSelector

- Uses of ModuleDependency.getRequested() to ModuleDependency.getSelector()

    • -2
    • +2
    ./InMemoryCachedModuleComponentRepository.java
  1. … 26 more files in changeset.
Add a `ModuleDependencyMetadata` subtype

Represents dependency information required for resolving a dependency from a

repository. DependencyMetadata sourced from the DSL is wrapped appropriately,

whereas DependencyMetadata sourced from an external module will have this type

natively.

Provides typed access to `ModuleComponentSelector`, making it easier to remove

uses of `DependencyMetadata.requested`

    • -2
    • +2
    ./InMemoryCachedModuleComponentRepository.java
  1. … 29 more files in changeset.
Added a new type `ResolveableArtifact` to use instead of public `ResolvedArtifact` when resolving and visiting the artifacts of a dependency resolution result.

    • -3
    • +3
    ./InMemoryCachedModuleComponentRepository.java
    • -2
    • +2
    ./InMemoryModuleComponentRepositoryCaches.java
  1. … 28 more files in changeset.
Removed unnecessary synchronization, and use a consistent pattern to protect cached dependency metadata and artifact state.

    • -2
    • +2
    ./InMemoryModuleComponentRepositoryCaches.java
Create a `ResolvedArtifact` wrapper for each external artifact once per build invocation, rather than once per dependency graph per build invocation.

    • -0
    • +11
    ./InMemoryCachedModuleComponentRepository.java
    • -0
    • +7
    ./InMemoryModuleComponentRepositoryCaches.java
  1. … 14 more files in changeset.
Apply some synchronization around the external module metadata that is cached in-heap and used by multiple threads.

    • -1
    • +1
    ./InMemoryModuleComponentRepositoryCaches.java
In-memory caches can be accessed concurrently

This commit replaces the regular hashmaps with concurrent hash maps, as they can be accessed concurrently. It will not prevent the same

value from being computed twice, potentially, but it will prevent incorrect state being stored in the maps (or worse, infinite loops).

Do not discard the in-memory metadata fetching cost cache between invocations

    • -15
    • +10
    ./InMemoryCachedModuleComponentRepository.java
  1. … 1 more file in changeset.
Change the heuristic from "is metadata locally availble" to "is it cheap to get metadata"

The rationale behind this change is that answering the question "do you have it locally" can in itself be very expensive. It also causes problems

with repository chains, because it's not because one of them has it locally, that we're not going to hit the network (because order of repositories

matter).

This commit changes the question from "is it available locally" to "is is cheap to get the answer"? If it's cheap to get the answer, then we're going

to perform the request serially. If it's not, we're putting it in the queue of work to be done concurrently. When a repository chain is hit, we then

make the difference between the 3 cases:

- the repository has the metadata at hand, and it's authoritative (it's available locally and up-to-date)

- the repository doesn't have the metadata at hand, but it's relatively cheap to get it, or tell that it's missing

- the repository doesn't have the metadata at hand, and it's expensive to get it or tell it's missing

Then, we can tell that a chain is cheap if, and only if, one of the repositories can get the authoritative answer fast, before those who are expensive,

or, all repositories are cheap.

    • -19
    • +14
    ./InMemoryCachedModuleComponentRepository.java
  1. … 18 more files in changeset.
Re-introduce a global cache for locally available metadata, which only stores results if it was actually found

    • -6
    • +10
    ./InMemoryCachedModuleComponentRepository.java
Availability of metadata should be cached on a per-repository basis

    • -2
    • +16
    ./InMemoryCachedModuleComponentRepository.java
  1. … 1 more file in changeset.
Introduce cache for local metadata availability

This commit adds a cache for local metadata availability. Since we introduced the heuristic to determine if metadata

is available locally, we could hit the persistent store multiple times for the same module during a build, which

introduced a performance regression. This commit adds the ability to cache the answer of availability for a build,

making it possible to do a short check before returning.

    • -3
    • +4
    ./InMemoryCachedModuleComponentRepository.java
Make sure we only pre-emptively download metadata for nodes which are `Selected`

This commit fixes a performance regression due to the fact we were downloading metadata for nodes which might

be excluded or new. It also fixes a problem with the in-memory metadata cache not telling that the metadata was

available locally, introducing lookups on disk when not necesary.

    • -0
    • +8
    ./InMemoryCachedModuleComponentRepository.java
  1. … 4 more files in changeset.
Protect state in `InMemoryArtifactsCache`

Cache the result of component artifact resolutions in memory, as is done for other resolution results.

    • -5
    • +23
    ./InMemoryCachedModuleComponentRepository.java
    • -7
    • +2
    ./InMemoryCachedRepositoryFactory.java
    • -10
    • +3
    ./InMemoryModuleComponentRepositoryCaches.java
  1. … 10 more files in changeset.
Reworked the way that a component's artifacts are calculated, so that the artifacts are determined in a single batch rather than once per configuration. All implementations of this calculation were in reality independent of the requested configuration, either using a value from the component's metadata or producing a result that is fixed for all of the component's configurations.

This change avoids duplicating a bunch of work that is only required to be done once, for example, determining the identity of the main artifact for a maven module whose packaging is not 'jar' by performing a HEAD request.

    • -2
    • +8
    ./InMemoryCachedModuleComponentRepository.java
  1. … 40 more files in changeset.
Extracted a common superclass out of the implementations for several resolve results, merged read-only and mutable artifact resolve results.

  1. … 18 more files in changeset.
No need to take a defensive copy of module meta-data reused from in memory cache.

  1. … 2 more files in changeset.
Changed some dependency resolution types to use an immutable module meta-data view instead of a mutable view.

We still take defensive copies in a few places, as the implementations are not actually immutable yet.

  1. … 11 more files in changeset.
Rename `o.g.i.c.model.*{MetaData => Metadata}`

    • -4
    • +4
    ./InMemoryCachedModuleComponentRepository.java
  1. … 162 more files in changeset.
Rename `o.g.i.c.e.model.*{MetaData => Metadata}`

  1. … 113 more files in changeset.
Removed a bunch of TODO:DAZ comments that I'm not going to do anytime soon

    • -1
    • +0
    ./InMemoryCachedModuleComponentRepository.java
  1. … 33 more files in changeset.
Introduce ResolverResults#getResolvedArtifacts

+review REVIEW-5515

  1. … 40 more files in changeset.
Renamed ComponentRequestMetaData -> ComponentOverrideMetadata

    • -2
    • +2
    ./InMemoryCachedModuleComponentRepository.java
  1. … 35 more files in changeset.
Start to migrate away from requiring DependencyMetaData when resolving component from id

- Wrap DependencyMetaData with ComponentRequestMetaData to restrict access

    • -2
    • +3
    ./InMemoryCachedModuleComponentRepository.java
  1. … 23 more files in changeset.