🚀 go-pugleaf

In article <413@mn-at1.UUCP> alan@mn-at1.UUCP (Alan Klietz) writes:
>A recent column in one of
>the Unix trade mags reveals that the Vax 780 = 1 MIPS rule-of-thumb
>may be grossly overstated.
[...]
>Therefore one VAX 780 "MIPS" is approximately 0.44 "honest-to-god"
>VAX MIPS?  (HTGV MIPS?)   Or a 126% overestimate?

This is well known.  I suspect you will get more than one reply (beside
this one) reiterating the story about how the DEC types benchmarked the
11/780 against a then-current 370 which IBM was calling a 1 MIPS machine,
found it to run about the same speed, and so for marketing purposes called
the 11/780 a "1 MIPS" computer.  Thus the "MIPS" referred to are supposed to
be native 370 MIPS, not native Vax MIPS.

This, unfortunately, is also not true, at least in my experience.  I have
found that you can match benchmark results on a 370 and a Vax pretty well
by multiplying the IBM-reported "MIPS" number by 1.8 or so (i.e. a 13 MIPS
3090 goes faster than one would otherwise be led to believe).  I really
think some marketeer at DEC just made up the 1 MIPS number so Vaxes would
look better against the IBM 370 competition.  The fact that trade magazines
are just getting around to realizing this shows what a good idea it was.

All of which matters not at all, since if you simply define a 780 to be
1 "MIPS" and measure everything against it, it all works out in the end
for many practical purposes anyway.

Dennis Ferguson
University of Toronto

>In article <413@mn-at1.UUCP> alan@mn-at1.UUCP (Alan Klietz) writes:
>>A recent column in one of
>>the Unix trade mags reveals that the Vax 780 = 1 MIPS rule-of-thumb
>>may be grossly overstated.

Omri Serlin reported that according to DEC data, A VAX 780 executes
about 470 thousand instructions per second for "typical" workloads.

In article <1988Mar14.002026.3977@gpu.utcs.toronto.edu> dennis@gpu.utcs.toronto.edu (Dennis Ferguson) writes:
>
>This is well known.  I suspect you will get more than one reply (beside
>this one) reiterating the story about how the DEC types benchmarked the
>11/780 against a then-current 370 which IBM was calling a 1 MIPS machine,
>found it to run about the same speed, and so for marketing purposes called
>the 11/780 a "1 MIPS" computer.  Thus the "MIPS" referred to are supposed to
>be native 370 MIPS, not native Vax MIPS.

I doubt if anyone really knows for sure where the axiomatic
"VAX 780 = 1 mips" came from.
I think it had two (possiblly related) points of origin.

1. The VAX 780 was evaluated as 1.06 mips in the August 2, 1982 Computerworld
   Annual Survey.  I spoke to someone at Computerworld at that time.
   He admitted the very approximate nature of their published mips ratings.
   He told me that they took performance claims from vendors, did some reason
   checking on them and published them with suitable statements about the
   approximate nature of such data.

   As far as I know DEC wasn't publicly making claims in 1982, that the
   780 was equal in performance to the canonical 1 mip IBM machine,
   the 370/158 Model 3. (I don't know the instruction rate of the 158-3).

2. When systems other than VAXen entered the Unix marketplace, their
   performance was characterized relative to a VAX.  The VAX was the obvious
   and convenient unit.
   Since vendors threw around mips claims like confetti on New Year's Eve,
   mips was a unit that everyone understood (or so someone thought).
   So if new Unix box had twice the performance of a VAX 780,
   it was called a 2 VAX mips box while a 780 was a 1 VAX mips box.

>
>This, unfortunately, is also not true, at least in my experience.  I have
>found that you can match benchmark results on a 370 and a Vax pretty well
>by multiplying the IBM-reported "MIPS" number by 1.8 or so (i.e. a 13 MIPS
>3090 goes faster than one would otherwise be led to believe).  I really
>think some marketeer at DEC just made up the 1 MIPS number so Vaxes would
>look better against the IBM 370 competition.  The fact that trade magazines
>are just getting around to realizing this shows what a good idea it was.

Some people at Amdahl published data last year on benchmarking in the
IBM and Unix worlds.  They should understand this better than anyone, since
they sell and evaluate systems in both performance measurement worlds.
My recollection is (someone at Amdahl correct me where necessary) that if
an Amdahl system was X IBM mips then the same system would be about 1.5 X times
the performance of a VAX 780 (NOT VAX MIPS !!!).

>All of which matters not at all, since if you simply define a 780 to be
>1 "MIPS" and measure everything against it, it all works out in the end
>for many practical purposes anyway.
>
>Dennis Ferguson
>University of Toronto

Yes.  The Unix performance measurement world does this.  Usually we state
performance as relative to a VAX 780.  Unfortunately, even here there is room
for confusion.  Different versions of Unix have different compilers and
(obviously) kernels which have different performance characteristics.
VAXes often perform better with VMS compilers.  Which is the "real" VAX
to measure against?

The Unix marketing world has simplified X times the performance of a
VAX 780 to X mips.


Steve Schlesinger
NCR Corporation


Disclaimer:	This is my opinion, not that of NCR Corporation.

In article <413@mn-at1.UUCP> alan@mn-at1.UUCP (Alan Klietz) writes:
>We all know what a VAX "MIPS" is, right?  But have you ever
>bothered to actually measure it?  DEC did.  A recent column in one of
>the Unix trade mags reveals that the Vax 780 = 1 MIPS rule-of-thumb
>may be grossly overstated.
>
>Therefore one VAX 780 "MIPS" is approximately 0.44 "honest-to-god"
>VAX MIPS?  (HTGV MIPS?)   Or a 126% overestimate?
>
  I offer a tiny bit of (probably well-known) historical perspective.

  Many moons ago, in the dim ages of the past, ITEL's System Development
  Division performed some measurements on an IBM 370/158 clone (AS-5)
  using an event counter driven by "EndOP" (microcode end-of-instruction)
  which yielded 1.08 MIPS over a 36-hour period.  Hourly rates varied
  between 1.01 and 1.20.    Common industry usage rated the 158 at
  one MIP.  It also rated the 158 at 2X a 780.

  From this usage comes the old rule-of-thumb that an IBM MIP is 2X
  a VAX MIP.  This rule is still in common use among business oriented
  computer publications.

  Since this group is primarily interested in scientific usage rather
  than business usage, perhaps the continued use of VUPS is justified.
  I don't want all the flamage that would arise from proposing the use
  of IBM MIPS.

>--
>Alan Klietz
>Minnesota Supercomputer Center (*)
>1200 Washington Avenue South
>Minneapolis, MN  55415    UUCP:  alan@mn-at1.k.mn.org
>Ph: +1 612 626 1836       ARPA:  alan@uc.msc.umn.edu  (was umn-rei-uc.arpa)
>
>(*) An affiliate of the University of Minnesota


--

Len Mills ...
{uunet,ucbvax!sun,lll-lcc!lll-tis,altos86,bridge2}!elxsi!len

In article <413@mn-at1.UUCP> alan@mn-at1.UUCP (Alan Klietz) writes:
:		In October 1987, one of the authors [of the DEC
:	study] presented a paper on the performance of the VAX
:	8800, a dual-processor machine in which each processor
:	is six times as fast as the 11/780, by DEC's official
:	reckoning.  This paper reported on some preliminary
:	measurements that showed the 8800 processor to have
:	a cycle per average instruction (CPAI) rate of 8.4.
:	At cycle time of 45ns, this translates into a MIPS rating
:	of 2.65.

	Which goes to show that 1 8800 MIPS != 1 11/780 MIPS.  Not suprising,
given the different memories, busses, caches, etc between an ancient old
(but ubiquitous) Vax 11/780 and an 8800.

	That's why people specify 11/780 Vax MIPS, not VAX MIPS.

     //	Randell Jesup			      Lunge Software Development
    //	Dedicated Amiga Programmer            13 Frear Ave, Troy, NY 12180
 \\//	beowulf!lunge!jesup@steinmetz.UUCP    (518) 272-2942
  \/    (uunet!steinmetz!beowulf!lunge!jesup) BIX: rjesup

(-: The Few, The Proud, The Architects of the RPM40 40MIPS CMOS Micro :-)

The poster of the referenced article wondered about how many MIPS
a VAX 780 ran.

Much of the computer industry uses MIPS as a metric for measuring computer
processor power; and uses it to compare processors from different families
and different manufacturers. If one wasn't interested in knowing how a
processor from DEC fared compared to one from, say, IBM, the comparison
of a 8600 to a 780 would be sufficient information.

Vendors claim that computing MIPS on machines by counting actual instructions
executed per second is unfair, because their machine architecture has
instructions which do more work than other machine architectures.
This sounded like bull to me; but it's not.

The number of machine instructions compiled to run the Dhrystone 1.1
benchmark on a VAX is considerably less than the number of IBM 360/370
instructions needed _for the same work_. And it's important to measure
work/second rather than something artificial. The Dhrystone 1.1 IBM
instructions appeared rather optimal to my trained eye. I don't have
the expertise to say how optimal the VAX instructions were.

I counted the IBM instructions, since I believe the comparisons to MIPS
started there many years ago and that this is what most of the industry is now
comparing. If you multiply this number by the ratio of the Dhrystones of two
machines, you can develop a IMIPS number how many IBM instructions worth of
Dhrystone work a processor does in a second. By running through the benchmark
compiled by Dhrystone 1.1, including subroutines, calculating how many times
each path is executed, I came up with 593 IBM instructions per Dhrystone 1.1
for the register variant.

My best Dhrystone 1.1 measurement for our 780 was 1628. My calculation is
that this is .965 IMIPS.

The VAX in fact executed a lot fewer instructions to complete the Dhrystones
than the IBM computers did. This seemed to be due to the multiple operand
instructions, with multiple addressing modes, that tended to replace
{load, figure, store result} sets of instructions with just one VAX
instruction. I would imagine the difference in measuring a RISC machine
in it's native MIPS would demonstrate more dramatically the difficulty
of using raw instructions/second measures in comparing different computer
architectures. I don't believe in honest-to-goodness-mips.

Note that the above IMIPS comparison assumes that Dhrystones are typical of
the workload one is interested in running on the machines in question, and
also assumes the level of optimization produced by the compilers used in
the benchmark. Your workload or compilers may vary. My results were with
the 4.3 cc compiler, and a MVS c compiler. Other results:
Computer	Best Dhrystone 1.1 Perf.	IMIPS
785  		2018        			1.20 IMIPS
4341-2 		2407     			1.43 IMIPS
8530		7065				4.19 IMIPS
4381-24 	5747    			3.41 IMIPS (each processor)

John Gruber gruber@andy.bgsu.edu tut!bgsuvax!gruber
--
John Gruber
University Computer Services       UUCP:..!cbosgd!osu-eddie!bgsuvax!gruber
Bowling Green State University     CSNET: gruber%bgsu@csnet-relay
Bowling Green, OH  43403-0125