Benchmark series on simple caching solutions in Java

Caching is a very common solution when you don't want to repeat CPU intense tasks. The last days I was benchmarking options to do caching with ConcurrentHashMap. In this blog I publish the first results. I have used Heinz Kabutz' Performance Checker to do this. I also added some features based on my readings of this article series.

My Conclusions up front

I am testing three different cache implementations: "Check null", "check map" and "putIfAbsent" cache. The code is listed below the results. Also, I am using three different cache sizes: 10 (small), 100000 (large) and 1000000 (very large) possible key values. Again: The 10 units cache can have 10 different key values. The 100000 units cache can have 100000 different key values etc.

None of the implementation options show significant performance differences assuming cache size is equivalent. This is disappointing (!!) but also good to know. It's also a little surprising I think, 'cause for example "putIfAbsent" cache appears to be more complex then the others. All the solutions seem to decrease logarithmical in performance when the cache increases to very large sizes. The main reason for that should be the increased time for cache initialization, 'cause in my test harness I start with an empty cash and a fixed set of possible key values (i.e. 10, 100000 or 1000000). I'll come up with a benchmark series for fully initialized cache later.

Knowing what I know now, I would carefully recommend to use the "putIfAbsent" cache solution. That's because it has equivalent performance but gives you great flexibility to design the behaviour of your cache in highly concurrent scenarios. See the pattern solution of my last article about multithreading as an example of more complex use cases.

If you're interested in the test harness take a look at the implementations: CacheSolution_CheckMap.java, CacheSolution_CheckNull.java and CacheSolution_PutIfAbsent.java. I appreciate your comments very much!

My JVM was in mixed JIT mode and with -server option set. OK, let's go into this!

Here are the results

5 test runs each / 500 ms each test run
CL before/after = Classes Loaded before and after test harness
JIT before/after = Total JIT time before and after test harness
Small cache = 10 units
Large cache 100000 units
Very large cache = 1000000 units

Check null cache

Check map cache

The putIfAbsent cache