Use separate user home for each version under performance test Until now, each performance test was using a separate test directory, but the different versions under test were all using the same directory.
This meant that therer was a consistent bias depending on the order in which the versions were executed.
Reduce memory for performance scenarios The scenarios should only have an amount of memory that is "reasonable" for what they are doing. This serves two purposes. It allows us to detect large memory regressions, as a reasonable upper limit will lead to lots of GC time if that limit is breached. It also makes test results more predictable, as too much memory means that many test runs will not need garbage collection at all while other test runs will have large GC cycles.