Clone
Stefan Oehme
committed
on 20 Mar 18
Use u-test to determine performance regressions
Instead of an arbitrary threshold we now use the Mann-Whitney U test
to calculate how confid… Show more
Use u-test to determine performance regressions

Instead of an arbitrary threshold we now use the Mann-Whitney U test

to calculate how confident we can be in the results.

For now we'll require a 99% confidence before reporting a regression.

We can tweak that value based on whether we miss too many

regressions or report too many non-regressions.

Show less

master + 428 more