The execution time of the junit test varies every time. What for?
I have a set of 196 methods . The execution time of these test boxes changes every time I run it. It was launched in a controlled environment. Let's say for garbage collection I include null
in teardown()
.
Every time before running tests, I also make sure that CPU usage, memory usage, disk space, system load are the same for every run.
Also, the time change is not in any particular order. I need to know why we don't get s table execution times when we run the same tests again?
I made 93 cases stable by incorporating a warm-up period into the class. Other cases involve connecting to a database (reading data or updating data in a database). Is it possible to have the same execution time every time I run these test windows. (Runtime refers to the runtime of the junit test program)
source to share
First of all there are problems with Java:
- You need to warm up the JVM, your tests are interpreted as bytecode and are at the mercy of the JVM. This means running the same test over a thousand times during the same run.
- JUnit tests are not measured with great precision. In fact, it is almost impossible to get an accurate performance reading, even if the libraries are specifically built for this. This is why the average number of several samples is usually accepted.
However, this and others suggested by Reto are what might cause Java to be rejected. Where the difference in milliseconds is greater than expected. For example, create a unit test that takes a thread and puts it to sleep for 10ms. Observe how you get results from 7ms to 13ms to 17ms or more. It's just not a reliable way to measure things.
If you are connecting to the network, loading data into a database, etc. I can't speak for this, but you need to consider the variance of these systems as well.
I would suggest breaking the three tests with the highest variance into smaller blocks. Try and isolate where your biggest bottleneck is, then focus on optimizing that operation or set of operations. I think the database connection takes the longest amount of time, next to which the query will probably run. But you must isolate the measurement of these operations to be sure.
source to share