(time (dotimes [_ 10000] (my-function arg1 arg2 arg3))

I normally end up writing something equivalent to that all the time to test reflection vs. type-hinted Clojure code, subtle tweaks and other things. Well, I finally got fed up extrapolating the information I wanted from the output of time (and time didn’t expressly return a number I could do anything with, just printed a string), so I wrote a small library for doing benchmarks a number of times. I called it CBench because I couldn’t think of anything that sounded Clojure-ish (bench-jure? clo-bench? etc).

Anyway, jumping straight in, here’s how it’s used, let’s say you have a couple of functions you want to test (super-simple examples):

(defn inc-no-hint [n] (inc n))
(defn inc-with-hints [#^Integer n] (inc n))


With cbench, you can perform benchmarking with a number of iterations. CBench provides 2 functions: cbench and cbench-pp. cbench-pp performs exactly like cbench, but prints a nice display instead of returning a vector of results. In this example, we’ll use cbench-pp:

(use 'cbench) ; cbench.jar must be in your classpath
(cbench-pp 10000 #(inc-no-hint (rand-int 100)))
(cbench-pp 10000 #(inc-with-hints (rand-int 100)))

Run this in a REPL (or a script), and you’ll see this output:

-----------------------------------
| avg    | 0.0161 ms
| min    | 0.0 ms
| max    | 26.0 ms
| stddev | 0.17861615575831605 ms
| total  | 570 ms
-----------------------------------
-----------------------------------
| avg    | 0.0012 ms
| min    | 0.0 ms
| max    | 8.0 ms
| stddev | 0.04897999523196905 ms
| total  | 238 ms
-----------------------------------

CBench gives the average time a run took, minimum, maximum, standard deviation and total time for running 10000 runs of each function. In this example we can see that type-hinting has made a > 10x improvement in average run time (note: this is a contrived example which doesn’t actually mean much other than hinting is faster than non-hinting).

If we wanted only the results as numbers, use the cbench function, in a REPL you can see the output:

user=> (cbench 10000 #(inc-with-hints (rand-int 100)))
[0.0032 22.0 0.0 0.07995998518952234 52]

The result is a vector (in the same order as the pretty-print version).

If you’re interested, the library is available from Clojars.org here, you can get just the jar, or you can check out the source code at the Github project page. Feel free to drop me an email, IM or github message if you like/dislike the library.

Oh yea, and on a non-technical note, I started a Tumblr blog so I can paste things I find interesting of a non-technical sort. If you’re into that sort of thing, feel free to check it out.