When benchmarking, it is all too easy to use the Common Lisp [time][time] macro to see how long something takes. For example,
(time (dotimes (i 1000) (do-my-function)))
There are at least two problems with this simple approach. Firstly, time doesn’t make it easy to record the values it computes; secondly, if your function is very fast, you may end up timing how quickly your lisp execute dotimes rather than measuring what you want. A better way to approach this sort of question is to ask “how many times can I execute my function in, e.g., 5-seconds?” The only downside of this approach is that it takes a little more code to set things up. Once you have the code, however, Bob is your uncle. [LIFT][] includes a small (and sadly under-documented) selection of functions and macros to help you benchmark and profile your code. Two of these are:
(defun count-repetitions (fn delay)
"Funcalls `fn` repeatedly for `delay` seconds. Returns
the number of times `fn` was called. Warning: the code
assumes that `fn` will not be called more than a fixnum
number of times."
...)
and
(defmacro while-counting-repetitions ((&optional (delay 1.0)) &body body)
"Execute `body` repeatedly for `delay` seconds. Returns
the number of times `body` is executed per second.
Warning: assumes that `body` will not be executed more
than a fixnum number of times. The `delay` defaults to
1.0."
...)
The main difference between the two variants — well, aside from one being a macro and one a function — is that count-repetitions must funcall your function whereas while-counting-repetitions includes your code in-line.
Let’s use them to look at three different solutions for the perennial “I want to turn a string into a keyword” problem. I’m also going to require that all of the strings be lowercased (just because). One solution uses read-from-string
(defun form-keyword-1 (name)
(read-from-string (format nil "~(:~A~) " name)))
another uses intern. We’ll look at two variants: the first looks up the keyword package each time; the second does it just once:
(defun form-keyword-2a (name)
(intern (string-downcase name) (find-package '#:keyword)))
(defun form-keyword-2b (name)
(intern (string-downcase name)
(load-time-value (find-package '#:keyword))))
Lastly, let’s see if there is any cost to calling intern unnecessarily by adding a variant that first uses find-symbol and only calls intern if it must.
(defun form-keyword-3 (name)
(let ((downcased-name (string-downcase name))
(keyword-package (load-time-value (find-package '#:keyword))))
(or (find-symbol downcased-name keyword-package)
(intern downcased-name keyword-package))))
Here’s what it looks like to try a simple test. We’ll use 5-seconds as our delay.
cl-user> (lift:count-repetitions
(lambda ()
(form-keyword-1 "hello")
(form-keyword-1 "butter"))
5.0)
57214.8
So form-keyword-1 (the read-from-string variant) can run about 57,000-times a second (this is under ACL in EMACS under Slime). The other three variants give counts of:
ACL SBCL
1: 57,215 37,354
2a: 689,888 329,896
2b: 986,085 447,137
3: 1,028,928 461,868
(Note: I’m using a somewhat out of date SBCL so these numbers shouldn’t be used to compare these two Lisps).
That’s a pretty huge difference on both of these Lisps. Allegro especially benefits from the load-time-value resolution of the keyword package and both Lisps should a slight improvement when calling find-symbol first (which actually surprised me. I would have thought that intern would have handled that logic already.)
Now lets see if funcalling is doing anything to change the results by switching over to the while-counting-repetitions macro:
cl-user> (lift:while-counting-repetitions (5)
(form-keyword-1 "hello")
(form-keyword-1 "butter"))
57633.6
Here’s the table
ACL SBCL
1: 57,634 37,960
2a: 686,941 328,316
2b: 992,373 451,186
3: 1,050,546 448,714
In this example, at least, the macro and the function appear to have no advantages one over the other.
