[Date Prev] [Date Next] [Thread Prev] [Thread Next] Date Index Thread Index Search archive:
Date:Mon Jan 3 13:04:39 1997 
Subject:Poplog Benchmarks on new machines 
From:A . Sloman 
Volume-ID:970103.01 

This message reports recent speed tests using Poplog running on
Digital Alpha stations (model 500/400 and model 255/233) and on
Sparc20/50 workstation upgraded with twin 180Mhz hypersparcs.

CONTENTS

 -- INTRODUCTION
 -- MULTI-PROCESSING TESTS (Alpha 500/400 and 180 Mhz Hypersparc)
 -- Dec Alpha 500/400 128 Mbytes memory, 1 CPU, local disks only
 -- Sun SS20/50 with 180 Mhz Hypersparc+256 Mbytes. 2CPUs. Files via NFS
 -- BENCHMARKS FOR VARIOUS MACHINES
 -- EXTRA TEST: Rebuilding saved images

-- INTRODUCTION
We have recently acquired some Dec Alphastations in Birmingham and
upgraded one of our SS20/50s with two of the latest Ross Hypersparc
modules (180Mhz). Poplog timing tests are appended.

Since prices for the latest hypersparc modules have fallen dramatically
they provide a good upgrade route for Sparc 10 or Sparc 20 machines
and also 600MP servers, if you want to keep old machines but increase
their speed. We had previously upgraded suns with 125Mhz and 66Mhz
hypersparcs a year ago and two years ago. The latest 180Mhz hypersparcs
are even cheaper and faster, and on my tests outperform 168 Mhz
Ultrasparc 2, as shown below. We get our Hypersparcs from Sellar
Performance, who give an educational discount:

    Stellar Performance Ltd
    Oxford House,
    24 Oxford Road North
    London W4 4DH
    Phone 0181 987 9922
    Fax   0181 987 9966

Contact: John Cheshire john@stellarperformance.co.uk

We have also recently acquired a lot of Digital AlphaStations (255/233
with 64Mbyte or 196 Mbytes for UG and MSc teaching and research
students), and one of the more powerful 500/400 Alphastations for a
research project. I've also run my Poplog benchmarks on all these.

Alpha speeds are very impressive compared with Suns. I've not been able
to test the latest HP and SGI machines, so I don't know how well they
would compare.

Summaries are appended, including results for some older machines for
comparison purposes. I've included results of an old HP test, so that
anyone who is interested in Poplog on HP can compare published speed
comparisons with the results for the HP735.
Usually poplog performance scales up very roughly like SpecFp and
SpecInt numbers (Spec92 or Spec95). See
    ftp://ftp.cdf.toronto.edu/pub/spectable
    http://www.specbench.org/

At the end of this file I've added times taken to rebuild our 12 local
saved images on some of our configurations. They are not strictly
comparable, since different file-server connections were involved.

-- MULTI-PROCESSING TESTS (Alpha 500/400 and 180 Mhz Hypersparc)
One of the most impressive results came from multi-processing tests on a
single CPU DEC Alpha station (Model 500/400)

I ran the prolog benchmark alone, then with two processes in parallel,
then 4, then 8, 12, 16 and 20, after which I hit a per user process
limit on the alpha (because of all the shell scripts involved). Here are
the times to completion for all the tests. I was able to run more
processes on the Sun. Results for the twin 180 Mhz hypersparc on a
Sparc 20/50 are shown below, after the Alpha results.

-- Dec Alpha 500/400 128 Mbytes memory, 1 CPU, local disks only

Number of   start time         end time         Elapsed time   Secs per
processes                                          (Secs)       process
1       Sun Oct 27 00:40:01  Sun Oct 27 00:40:04       3       (approx)
2       Sun Oct 27 00:12:26  Sun Oct 27 00:12:31       5          2.5
4       Sun Oct 27 00:12:52  Sun Oct 27 00:13:02      10          2.5
8       Sun Oct 27 00:13:43  Sun Oct 27 00:14:03      20          2.5
12      Sun Oct 27 00:14:42  Sun Oct 27 00:15:12      30          2.5
16      Sun Oct 27 00:15:45  Sun Oct 27 00:16:25      40          2.5
20      Sun Oct 27 00:29:07  Sun Oct 27 00:29:58      51          2.55

These tests were run on a free-standing machine. I have not tried this
since the machine was integrated into our network running NFS, NIS etc.
This would introduce delays in fetching files.


-- Sun SS20/50 with 180 Mhz Hypersparc+256 Mbytes. 2CPUs. Files via NFS

Similar tests for the SS20/50 upgraded with twin 180Mhz Hypersparc.
Files were obtained remotely via NFS from a separate departmental server
(SparcCenter1000).

Number of   start time         end time         Elapsed time   Secs per
processes                                         (Secs)       process
1       Mon Dec 30 00:18:37  Mon Dec 30 00:18:45     8           8
2       Mon Dec 30 00:21:15  Mon Dec 30 00:21:24     9           4.50
4       Mon Dec 30 00:08:08  Mon Dec 30 00:08:26    18           4.50
8       Sun Dec 29 23:44:17  Sun Dec 29 23:44:53    36           4.50
12      Sun Dec 29 23:52:36  Sun Dec 29 23:53:29    53           4.42
16      Sun Dec 29 23:58:16  Sun Dec 29 23:59:27    71           4.44
20      Mon Dec 30 00:02:39  Mon Dec 30 00:04:08    89           4.45
24      Mon Dec 30 00:06:20  Mon Dec 30 00:08:07   107           4.46
30      Mon Dec 30 00:12:29  Mon Dec 30 00:14:42   133           4.43

NB: the time per process is achieved only because there are two
processors. With one processor it would be approximately double (i.e.
about 9 secs compared with 2.5 secs on single processor Alpha 500/400,
with local file store.)

-- BENCHMARKS FOR VARIOUS MACHINES

Here's the Poplog revised benchmark table. Note that these are top
speeds, not averages.

The HS125 and HS180 are the same machine first with a dual Hypersparc
    (125 Mhz with 256Kbyte cache) module, then after upgrading to
    twin Hypersparc (180Mhz with 512Kbyte cache)
    Both ran in a SS20/50 monitorless workstation, with 256 Mbytes ram.

USP1  is a single processor UltraSparc 1 140MHz CPU, with 128Mb ram.
USP2  is a twin processor UltraSparc 2e  170MHz CPU, with 256Mb ram.

"nosey" is a DEC Alphastation 255/233 with 64 Mb ram.
"isdugp" is a dual processor Alpha Server 8200/300Mhz with 512 Mb
    (Large expensive machine in Computing Centre at Birmingham)
"genetic" is an AlphaStation 500/400 with 128 Mbytes memory.


                   SUPER   HYPERSPARC       ULTRASPARC  <----DEC ALPHA------>
                   SPARC <-------------->  <--------->  <----DEC Unix ------>
                    Sun  Sun   Sun    Sun  USP1   USP2  nosey  isdugp  genetic  HP    Sun*
                    SS10 Ross  Ross   Ross Ult-1 Ult-2  alpha   Alpha  Alpha   HP-UX  SS2
                     /41 HS66 HS125  HS180 140   168    255/233 8200/  500/400  735
                         mhz   mhz    mhz  mhz   mhz      mhz  300mhz  400mhz         SunOS
                        SunOs Sol2.4 Sol2.4 Sol2.5             4Mcache 2M cache
-------------------------------------------------------------------------------------------
Prolog test KLips   238  335   563    775  620   729    590     1240    1559    387  117.0
(i.e. Naive rev)               590?                     496#            2067$
-------------------------------------------------------------------------------------------
Pop-11
Factorial(1000)
three times (Secs)
    recursive       0.46 0.38 0.22   0.17  0.27  0.23  0.06     0.03    0.01    0.11  0.81
    iterative       0.45 0.38 0.21   0.16  0.27  0.22  0.06     0.03    0.01    0.11  0.78
nine times (secs)
    recursive                 0.67   0.53  0.88  0.74  0.23     0.1     0.06
    iterative                 0.64   0.5   0.85  0.73  0.21     0.1     0.06
-------------------------------------------------------------------------------------------
Pop-11
Floating point
50000 times
    single (secs)   0.19 0.13 0.07   0.05  0.09  0.07  0.05     0.01    0.03    0.13  0.38
    double          0.22 0.16 0.09   0.06  0.11  0.08  0.08     0.03    0.03    0.17  0.43
500000 times
    single (secs)             0.78   0.55  0.94  0.78  0.55     0.41    0.3
    double                    0.95   0.68  1.04  0.88  0.8      0.5     0.38
-------------------------------------------------------------------------------------------
* SS2 runs significantly faster with weitek upgrade (60% or more)
? I cannot now replicate the original 590 Klips figure on our HS125.
  It now only reaches 563.
$ The higher KLIPS figure for the Alpha500/400 was on a standalone machine.
  Integration into our network reduced the top speed, presumably because
  part of the cache was permanently occupied by NFS/automounter, etc.
# The lower figure for Poplog prolog on the Alpha 255/233 was obtained
  in Poplog V15.01. The upper one in Poplog V15.5. The difference between
  V15.01 and V15.5 did not affect performance on the 500/400 so much.

NOTE: the 125 Mhz and 180 Mhz hypersparcs were running Solaris 2.4 in
an SS20/50. The 66Mhz hypersparc was running SunOS 4.1.3 in an SS10 or
SS 672.

NB some of the tests need to be updated to give more meaningful figures for
the faster processors.

-- EXTRA TEST: Rebuilding saved images

Time taken to rebuild twelve local Poplog saved images (including new
startup.psv with rc_graphic and several other local extras included):
boole.psv   clisp.psv   eliza.psv   kitchen.psv   logic.psv     msblocks.psv
plog.psv    pml.psv     poly.psv    prolog.psv    startup.psv   xved.psv

[NB the images include links to external libraries (e.g. motif) and
these processes are different on different operating systems.]

NAME        MACHINE               TIME
genetic     Alpha 500/400       =  47 secs  = 0 min 47 secs
    (measured before connection to the network.)
genetic     Alpha 500/400       = 149 secs  = 2 min 29 secs
    (using NFS link to departmental file server. Also with one low
    priority CPU-bound process on the machine).

isdugp      Alpha 8200/300      = 100 secs  = 1 min 40 secs
   (multi-user machine in computing centre)

[The remaining machines depended on access to files on the CS server.]
christopher- Alpha 255/233:     = 200 secs  = 3 mins 20 secs
    robin    (Poplog V15.5)

wallace     125 Mhz Hypersparc  = 279 secs  = 4 mins 39 secs
wallace     180 Mhz Hypersparc  = 232 secs  = 3 mins 52 secs
            (Upgraded SS20/50)

fat-        66 Mhz Hypersparc  = 457 secs  = 7 Mins 37 secs
 controller  (Upgraded 600MP)

Note: Alphas tested using Poplog V15.5(Beta). All others using
Poplog V15.01

Aaron
===
Aaron Sloman, ( http://www.cs.bham.ac.uk/~axs )
School of Computer Science, The University of Birmingham, B15 2TT, England
EMAIL   A.Sloman@cs.bham.ac.uk
Phone: +44-121-414-4775 (Sec 3711)       Fax:   +44-121-414-4281
           DO NOT SEND ANY FORM OF ADVERTISING TO THIS ADDRESS