On Wed, 13 Oct 2004 05:53:07 +0000 (UTC), bfulgham@debian.org wrote:
>On 2004-10-12 10:48:02 -0700 Aaron Sloman wrote:
>> And if you know what you are doing (and are careful) you
>> can get even better results. But never as good as the
>> best compiled lisp systems, or C.
>
>Well, that's a shame. But Poplog exists in the same problem space
>as Erlang and Mozart/Oz, and so is targeted at a different
>kind of problem. Unfortunately, the shootout tests are not good
>at testing these categories of problems. I hope to change that
>assuming I can identify a set of problems that are:
> o Easy to implement in a variety of languages (usually this
> equates to being short programs.)
> o Can be scaled across a variety of inputs, allowing a range
> of input tests to be run.
> o Allow the use of a known "good" answer/output that can be
> used to judge (in an automated fashion) that two implementations
> of a test have arrived at the correct answer.
If this kind of thing interests you, try and borrow a copy of
the book "Performance and Evaluation of Lisp Systems" by Richard
P Gabriel, MIT Press, 1985. It has about 80 pages discussing Lisp
implementation and 12 different Lisp implementations, and then
about 200 pages of benchmark programs, including theorem provers
and a FFT test. (I got my copy when the British Library sold it,
as part of its regular clearing out of unwanted books.)
The first few test problems are very short, e.g. TAK, which tests
function calling and recursion and in pop11 would be something like:
define tak(X, Y, Z);
if not(Y < X) then
Z
else
tak(tak(X-1, Y, Z),
tak(Y-1, Z, X),
tak(Z-1, X, Y))
endif
enddefine;
tak(18, 12, 6)=>
The book says
When called on the above arguments, TAK makes 63,609 function calls
and performs 47,706 subtractions by 1. The result of the function
is 7. The depth of recursion is never greater than 18. No garbage
collection takes place in the MacLisp version of this function
because small fixnums are used.
Note that MacLisp is nothing to do with Apple Macintosh: they hadn't
been invented when the book was written!
From the table of about 28 implementation/machine/OS combinations
they benchmarked this on (page 85), it seems (PSL) Portable Standard
Lisp on a VAX750 running VMS took 1.37 seconds, but PSL on a Cray
supercomputer took 0.04 seconds.
For comparison, pop11 on my antique (five year old) home PC takes
about 1.1 milliseconds, or 0.001 seconds. (I had to run it a 1000
times to get something long enough to measure.) I imagine a modern
PC would be ten times faster again.
Anyway, until we run off the end of Moore's Law, and speeds stop
doubling every couple of years, the difference between the speed
of a C prog and a Lisp or Pop11 prog isn't really significant for
practical purposes.
I have my own, personal, benchmark called TRIB for measuring
function calls, simple arithmetic and recursion. Here's the lisp
code for it:
(defun trib (n)
(declare (optimize (speed 3) (safety 0) (space 0)))
(declare (fixnum n))
(if (< n 3) n (+ (trib (1- n)) (trib (- n 2)) (trib (- n 2)))))
(defun rating (n time)
(- n (log time 2)))
You time a call of TRIB with a suitable value of N. Increasing
N by 1 doubles the time it takes for the benchmark to run. So
suppose you call TRIB with N=25,
(time (trib 25))
on my antique PC running Allegro CL 3.0 on Windows 98, it takes 7.61
seconds, so the rating is
(rating 25 7.61)
which gives a rating of 22.1. On the same PC, Poplog pop11, running
under a variant of Linux,
define trib(n);
lvars n;
if n < 3 then
n
else
trib(n-1) + trib(n-2) + trib(n-2) ;;; NB, not n-3 on last call
endif
enddefine;
I get a time of 3.68 seconds for trib(25), or 7.35 for trib(26),
which gives a rating of 23.1, or almost exactly twice as fast.
Somewhere (but I can't find them) I have TRIB ratings for a whole
bunch of languages, compilers and operating systems, including
interpreted Basic on a Z88 computer.
I did find some figures, dating back a few years:
=============
27/4/92
Mac LC 17.83 Think C
Mac IIx 18.34 "
386 33MHz 18.46 Borland C++
" (float 32) 16.27 " (32 bit float)
" (float 64) 16.20 " (64 bit float)
SPARC 1 18.88 pop11 (fi_) optimised
" 20.16 cc
" 21.21 cc -O
" 17.83 pop-11 (normal +/-)
SUN 4/260 21.15 cc -O
Mac Classic 16.05 Think C (System 6.0.7)
=============
28/1/95
Mac LC III 16.88 MCL 2.0 (default)
" 17.83 " (full optim)
486 DX4-100 19.04 Franz Allegro CL 2.0
" 19.24 " (full optim)
Mac LC III 19.03 Metrowerks 7 (default)
486 DX4-100 21.21 Borland 4.5 (default)
=============
02/7/00
PentIII-350 25.35 Visual C++ 5.0
That last figure is the same PC I just ran the Lisp
and Pop11 benchmarks on.
I hope this was of some entertainment value!
Jonathan
P.S. For non-Lisp programmers, the rating is N minus
the log of the time to base 2, so rating would be
something like:
define rating(n, t);
n - (log(t)/log(2))
enddefine;
but I didn't test this. You should get more-or-less
the same rating whatever value of n you use, it just
takes longer.
--
Use jlc1 at address, not spam.
|