Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
First stab at parallelizing worker daemons

    • -205
    • +0
    ./internal/AbstractWorkerDaemonServiceIntegrationTest.groovy
    • -73
    • +0
    ./internal/WorkerDaemonExpirationIntegrationTest.groovy
    • -330
    • +0
    ./internal/WorkerDaemonServiceErrorHandlingIntegrationTest.groovy
    • -409
    • +0
    ./internal/WorkerDaemonServiceIntegrationTest.groovy
    • -154
    • +0
    ./internal/WorkerDaemonServiceLoggingIntegrationTest.groovy
  1. … 103 more files in changeset.
Change worker daemon logging test to spend less time in async polling

    • -4
    • +9
    ./internal/WorkerDaemonServiceLoggingIntegrationTest.groovy
Temporarily disable WorkerDaemonServiceLoggingIntegrationTest

    • -0
    • +2
    ./internal/WorkerDaemonServiceLoggingIntegrationTest.groovy
Fix flakiness in worker daemon expiration test

It doesn’t check that starting a worker daemon actually calls

`MemoryManager.requestFreeMemory()` but that is already covered by

WorkerDaemonExpirationTest.

    • -20
    • +25
    ./internal/WorkerDaemonExpirationIntegrationTest.groovy
Ignore the test for now

    • -0
    • +2
    ./internal/WorkerDaemonExpirationIntegrationTest.groovy
Don’t try to allocate twice the available system memory in test

It fails on Windows

Follow up to 32800d9 and #1048

    • -2
    • +2
    ./internal/WorkerDaemonExpirationIntegrationTest.groovy
Less frequently used idle worker daemons memory based expiration

Less frequently used idle worker daemons are expired when available system memory gets below a threshold.

Expiration strategy is triggered both by memory sampling and when the build daemon is about to fork a new process, or a new worker daemon. It will attempt to free enough system memory to fit the new process below the threshold according to it's fork options.

This implementation does an approximation on the amount of memory held by worker daemons, using their max heap setting instead of their committed memory. This will be fixed in subsequent changes.

    • -0
    • +66
    ./internal/WorkerDaemonExpirationIntegrationTest.groovy
  1. … 24 more files in changeset.
Adding tests for fork options on worker daemons

    • -1
    • +4
    ./internal/WorkerDaemonServiceErrorHandlingIntegrationTest.groovy
    • -0
    • +144
    ./internal/WorkerDaemonServiceIntegrationTest.groovy
More test coverage for worker daemon failure scenarios

- Check failures when instantiating the runnable fails

- Check failures when the wrong parameters are passed to the worker daemon

    • -12
    • +76
    ./internal/WorkerDaemonServiceErrorHandlingIntegrationTest.groovy
  1. … 4 more files in changeset.
Handle serialization exceptions gracefully in worker daemons

- Add StreamFailureMessage message type

- Have MessageHub receiver/dispatcher catch object stream errors, inject StreamFailureMessage, and continue processing instead of exiting

- Catch object stream errors in SocketConnection and throw a typed exception

    • -4
    • +28
    ./internal/AbstractWorkerDaemonServiceIntegrationTest.groovy
    • -0
    • +263
    ./internal/WorkerDaemonServiceErrorHandlingIntegrationTest.groovy
    • -68
    • +0
    ./internal/WorkerDaemonServiceIntegrationTest.groovy
  1. … 9 more files in changeset.
Move worker processes tests back in :core

Follow up on #943

Worker daemons are in :workers, but worker processes could not be

moved there because of dependencies to :core.

    • -116
    • +0
    ./internal/AbstractWorkerProcessIntegrationSpec.groovy
    • -28
    • +0
    ./internal/BrokenRemoteProcess.java
    • -33
    • +0
    ./internal/CrashingRemoteProcess.java
    • -211
    • +0
    ./internal/MultiRequestWorkerProcessIntegrationTest.groovy
    • -41
    • +0
    ./internal/NoCleanUpRemoteProcess.java
    • -158
    • +0
    ./internal/PathLimitationIntegrationTest.groovy
    • -197
    • +0
    ./internal/SingleRequestWorkerProcessIntegrationTest.groovy
  1. … 27 more files in changeset.
Fix issue with logging output arriving late

    • -0
    • +181
    ./internal/AbstractWorkerDaemonServiceIntegrationTest.groovy
    • -222
    • +2
    ./internal/WorkerDaemonServiceIntegrationTest.groovy
    • -0
    • +147
    ./internal/WorkerDaemonServiceLoggingIntegrationTest.groovy
More cleanup in WorkerProcessIntegrationTest

    • -1
    • +0
    ./internal/WorkerProcessIntegrationTest.groovy
Address review items

- don't register core classloader when registering incoming messages

- remove unused overload of ObjectConnectionBuilder#addIncoming

- cleanup WorkerDaemonServiceIntegrationTest

    • -1
    • +0
    ./internal/WorkerDaemonServiceIntegrationTest.groovy
  1. … 3 more files in changeset.
Fix SingleRequestWorkerProcessIntegrationTest after changed worker process logging behaviour

    • -1
    • +1
    ./internal/SingleRequestWorkerProcessIntegrationTest.groovy
Add coverage for log level settings reflected in worker processes'

    • -1
    • +1
    ./internal/AbstractWorkerProcessIntegrationSpec.groovy
    • -2
    • +30
    ./internal/WorkerProcessIntegrationTest.groovy
Add more test coverage for logging in worker processes

    • -1
    • +1
    ./internal/AbstractWorkerProcessIntegrationSpec.groovy
    • -0
    • +67
    ./internal/WorkerProcessIntegrationTest.groovy
  1. … 7 more files in changeset.
fix test `workerProcessStdoutAndStderrIsForwardedToThisProcess`

    • -4
    • +2
    ./internal/WorkerProcessIntegrationTest.groovy
  1. … 1 more file in changeset.
Ignore WorkerProcessIntegrationTest#workerProcessStdoutAndStderrIsForwardedToThisProcess for now

    • -1
    • +3
    ./internal/WorkerProcessIntegrationTest.groovy
Redirect LogEvent and StyledOutputEvent from worker processes to server

* use custom serializer

* Added test coverage

* TODO fix WorkerProcessIntegrationTest:workerProcessStdOutAndStdInIsForwardedToThisProcess

    • -1
    • +4
    ./internal/AbstractWorkerProcessIntegrationSpec.groovy
    • -0
    • +42
    ./internal/WorkerDaemonServiceIntegrationTest.groovy
    • -3
    • +3
    ./internal/WorkerProcessIntegrationTest.groovy
  1. … 13 more files in changeset.
Move workers into their own subproject

    • -0
    • +113
    ./internal/AbstractWorkerProcessIntegrationSpec.groovy
    • -0
    • +28
    ./internal/BrokenRemoteProcess.java
    • -0
    • +29
    ./internal/BrokenTestWorker.java
    • -0
    • +33
    ./internal/CrashingRemoteProcess.java
    • -0
    • +34
    ./internal/CrashingWorker.java
    • -0
    • +30
    ./internal/LoggingProcess.java
    • -0
    • +211
    ./internal/MultiRequestWorkerProcessIntegrationTest.groovy
    • -0
    • +41
    ./internal/NoCleanUpRemoteProcess.java
    • -0
    • +34
    ./internal/NotDeserializable.java
    • -0
    • +31
    ./internal/OtherRemoteProcess.java
    • -0
    • +158
    ./internal/PathLimitationIntegrationTest.groovy
    • -0
    • +50
    ./internal/PingRemoteProcess.java
    • -0
    • +54
    ./internal/RemoteProcess.java
    • -0
    • +197
    ./internal/SingleRequestWorkerProcessIntegrationTest.groovy
    • -0
    • +31
    ./internal/StatefulTestWorker.java
  1. … 108 more files in changeset.
Prevent usage of Worker Daemons from custom threads

When a task action starts a thread that calls execute() and the task

action blocks waiting for the thread to complete, we can deadlock due

to running out of leases, for example, if this is run with

--max-workers=1.

Given that we don't have great visibility about how tasks and threads

are related, we simply have execute() throw an exception if not used

from a thread we know about, ie a thread that does not have a build

operation associated with it.

    • -0
    • +41
    ./internal/WorkerDaemonServiceIntegrationTest.groovy
  1. … 1 more file in changeset.