Clever Geek Handbook

HikariCP: Moving from BoneCP and Analyzing Control Data

I am testing migrating from BoneCP to HikariCP.

Benchmark results ( HikariCP Benchmark ):

Benchmark (maxPoolSize) (pool) Mode Samples Score Score error Units
czhbConnectionBench.cycleCnnection 32 hikari thrpt 16 1033.071 38.893 ops / ms
czhbConnectionBench.cycleCnnection 32 bone thrpt 16 1648.954 42.610 ops / ms
czhbConnectionBench.cycleCnnection 32 tomcat thrpt 16 592.838 21.709 ops / ms
czhbConnectionBench.cycleCnnection 32 c3p0 thrpt 16 125.857 3.666 ops / ms
czhbConnectionBench.cycleCnnection 32 vibur thrpt 16 831.693 9.299 ops / ms
czhbStatementBench.cycleStatement 32 hikari thrpt 16 49983.959 4796.385 ops / ms
czhbStatementBench.cycleStatement 32 bone thrpt 8 7022.146 526.111 ops / ms
czhbStatementBench.cycleStatement 32 tomcat thrpt 16 28777.731 1011.316 ops / ms
czhbStatementBench.cycleStatement 32 c3p0 thrpt 16 3367.641 137.937 ops / ms
czhbStatementBench.cycleStatement 32 vibur thrpt 16 1739.060 45.451 ops / ms

I believe a better result is better, so:

  • Why is cycConnection lower in HikariCP than BoneCP?
  • Are the results of cycStatement good enough to transition to HikariCP given the results of theConnection loop?
  • What does the Error Score column mean?

Any tips, advice, etc. in these topics or related migration to HikariCP are very helpful to me.

Thank you so much

FIRST UPDATE :

@brettw, thank you for your suggestions and congratulations for a great job.

I did a full run last night with nohup./benchmark.sh and got the following results:

Benchmark (maxPoolSize) (pool) Mode Samples Score Score error Units
czhbConnectionBench.cycleCnnection 32 hikari thrpt 150 965.017 27.307 ops / ms
czhbConnectionBench.cycleCnnection 32 bone thrpt 150 1528.097 26.963 ops / ms
czhbConnectionBench.cycleCnnection 32 tomcat thrpt 150 473.702 8.068 ops / ms
czhbConnectionBench.cycleCnnection 32 c3p0 thrpt 150 108.527 1.591 ops / ms
czhbConnectionBench.cycleCnnection 32 vibur thrpt 150 740.603 14.556 ops / ms
czhbStatementBench.cycleStatement 32 hikari thrpt 60 47998.292 3112.217 ops / ms
czhbStatementBench.cycleStatement 32 bone thrpt 120 6210,647 110,100 ops / ms
czhbStatementBench.cycleStatement 32 tomcat thrpt 150 25919.389 437.232 ops / ms
czhbStatementBench.cycleStatement 32 c3p0 thrpt 150 3074.133 62.629 ops / ms
czhbStatementBench.cycleStatement 32 vibur thrpt 150 1517.554 25.027 ops / ms

Viewing the expanded output, I see this exception stack trace 6 times in the log:

Iteration 15: [com.zaxxer.hikari.benchmark.StatementBench.cycleStatement-jmh-worker-7] INFO com.zaxxer.hikari.pool.HikariPool - HikariCP pool HikariPool-0 is shutting
down.


java.lang.IllegalStateException: Attempt to remove an object from the bag that was not borrowed or reserved
        at com.zaxxer.hikari.util.ConcurrentBag.remove (ConcurrentBag.java:207)
        at com.zaxxer.hikari.pool.HikariPool.closeConnection (HikariPool.java:394)
        at com.zaxxer.hikari.pool.HikariPool.releaseConnection (HikariPool.java:223)
        at com.zaxxer.hikari.proxy.ConnectionProxy.close (ConnectionProxy.java:216)
        at com.zaxxer.hikari.benchmark.StatementBench $ ConnectionState.teardown (StatementBench.java:63)
        at com.zaxxer.hikari.benchmark.generated.StatementBench_cycleStatement.cycleStatement_Throughput (StatementBench_cycleStatement.java:100)
        at sun.reflect.GeneratedMethodAccessor1.invoke (Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke (Method.java:606)
        at org.openjdk.jmh.runner.LoopBenchmarkHandler $ BenchmarkTask.call (LoopBenchmarkHandler.java:204)
        at org.openjdk.jmh.runner.LoopBenchmarkHandler $ BenchmarkTask.call (LoopBenchmarkHandler.java:186)
        at java.util.concurrent.FutureTask.run (FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor $ Worker.run (ThreadPoolExecutor.java:615)
        at java.lang.Thread.run (Thread.java:745)

Finally, I answer other threads from your post:

  • pom.xml updated for fast and complete executions.
  • Unable to execute test after clean boot.
  • There is a Tomcat server with a web app. The computer was not under heavy load and did not perform a heavy process during test runs.
  • JDK version: jdk1.7.0_67
  • Linux - x86_64-redhat-linux-gnu
  • Processor Model - Intel (R) Xeon (R) CPU E5-2690 0 @ 2.90 GHz
  • Possibly an important issue: The test was running in a VM deployed with VMWare.

Thanks again

SECOND UPDATE : The problem with the migration has changed to another one related to virtualization and benchmarking, which is also interesting and useful.

Currently I have to focus on the virtual environment. When I had a little time, I will run tests in a real and live environment and I will present the results here. I'm sure these tests will show HikariCP as the best connection pool.

@brettw thank you for your support and your great attitude.

+4


source to share


1 answer


one of the authors of HikariCP. A couple of things to check. First, it looks like you were using the "fast" test mode, which we usually use to "smoke test" quickly. I recommend full launch. The above results look a little odd, at least on a loop test, since all pools should be showing 16 (in fast mode), but BoneCP only shows 8. Full startup takes quite a long time, but if you just want to compare the two pools, you can shorten the time a little, for example:

./benchmark.sh -p pool=hikari,bone

Second, edit the pom.xml file for the test and make sure it is using the latest version of HikariCP. It should now be version 2.2.5 (non-SNAPSHOT), but the captured pom.xml looks like it still has 2.1.1-SNAPSHOT.

The "estimate error" is a kind of standard deviation in the samples, it tends to creep higher if other processes are consumed on the processor. I recommend running the test after a clean boot, after the system has been "installed" for a minute or two, and make sure that extraneous processes (browsers, etc.) are not running during the test.

We'll be interested in updated results from your launch as we have yet to see HikariCP score lower than any pool. Can you also tell us what environment you are working in? Processor type / cores, OS, Java version and whether it is a virtual machine or not. Thank.

UPDATE:
How is it possible for an 8-core E5-2690 to come with such low clock rates? On my desktop i7 Haswell 4-core I get HikariCP scores of 23130.760 and bone scores 10378.450 versus 965.017 and 1528.097 on E5-2690?



I am curious (and researching) that the thing about VMWare Hypervisor throttling is worn. I admit that I am far from being an expert on VM Hypervisors. The HikariCP test will definitely punish your processor.

UPDATE2:
Reply to comment below. This problem is not, in fact, virtualization. Virtualization is great and is here to stay. The problem mainly boils down to the fact that the listing of micro- benchmarks on virtualized hardware is accurate or meaningful.

I would suggest that, in general, a library that focuses better on real hardware is likely to perform better on virilized hardware - even if the same tool cannot be used to measure it. This is why, above, I recommended using a -benchmarking macro tool like JMeter for the intended application. Try it with Library A (HikariCP) and Library B (BoneCP). This will be true for any two libraries.

So working micro-control harnesses that use things like hardware counters, etc. may very well yield weird results on virtualized hardware. This is probably a great topic for discussion on the VMWare forum.

For which library to use, run JMeter or a similar macro benchmarking tool or flip a coin.

+5


source






More articles:


All Articles