Clever Geek Handbook

The right balance between multiple iterations and the number of forks in JMH

I am looking into OpenJDK JMH to compare my code. It is my understanding that JMH wags multiple JVMs by default to protect the test from previously collected "profiles". This is explained very well in the sample code .

However, my question is what impact will I have on the result if I execute using the following two approaches:

1) with 1 fork, 100 iterations 2) with 10 forks, 10 iterations each

And which approach will give a more accurate result?

+3


source to share


2 answers


It depends. Several forks are required to estimate deviation from run-to-run, see JMHSample_13_RunTo_Run . Therefore, one fork is definitely worse. Then, if you ask which is better, a 10x100 or 100x10 run, it again depends on which is a more complex issue: run-to-run or mileage in perspective.



+4


source


It depends on how much the results vary by fork versus each iteration, which is workload specific.



If you need a rigorous statistical approach to figuring out this tradeoff, see Strict Benchmarking at a Reasonable Time (Kalibera, Jones) . Equation 3 gives the optimal values ​​for each level (in your case, this would be the number of forks to run and the number of iterations per fork). using the observed deviations between forks and between iterations.

0


source






More articles:


All Articles