*BSD News Article 74650


Return to BSD News archive

Path: euryale.cc.adfa.oz.au!newshost.anu.edu.au!harbinger.cc.monash.edu.au!news.mira.net.au!inquo!in-news.erinet.com!newsfeeder.sdsu.edu!news.sgi.com!olivea!spool.mu.edu!howland.reston.ans.net!EU.net!news.eunet.fi!sja
From: sja@tekla.fi (Sakari Jalovaara)
Newsgroups: comp.os.linux.networking,comp.unix.bsd.netbsd.misc,comp.unix.bsd.freebsd.misc
Subject: Re: TCP latency
Date: 25 Jul 1996 09:44:58 GMT
Organization: Helsinki University of Technology
Lines: 32
Message-ID: <SJA.96Jul25124458@poveri.tekla.fi>
References: <4paedl$4bm@engnews2.Eng.Sun.COM> <4sadde$qsv@linux.cs.Helsinki.FI>
	<31E9E3A7.41C67EA6@dyson.iquest.net> <4sefde$f0l@fido.asd.sgi.com>
	<4socfr$3ot@dworkin.wustl.edu> <4t63as$f2q@fido.asd.sgi.com>
NNTP-Posting-Host: poveri.tekla.fi
In-reply-to: lm@neteng.engr.sgi.com's message of 24 Jul 1996 21:07:40 GMT
Xref: euryale.cc.adfa.oz.au comp.os.linux.networking:46266 comp.unix.bsd.netbsd.misc:4160 comp.unix.bsd.freebsd.misc:24367

Re "null syscall benchmark"

> I'm also willing (and interested) to find a different syscall that just
> measures trap overhead, but I haven't seen one yet that I really like.
> The getppid() may be the best out there, though, it's hard to cache 
> that.  Thoughts?

It seems painfully clear that null syscalls are rather unusual beasts.
What is a "null syscall" in one system, isn't in another.

A null syscall benchmark seems to reflect precious little real-world
program behavior.  It is not useful in comparing operating systems or
even different versions of the same operating system.  Such a
benchmark exercises a small portion of a minute detail of an
artificially constructed non-real-world program.

So, as a thought: as a service to the free OS community, give it up.
The flame/usefulness ratio of null syscall benchmarks is damn near
infinite.

If you are into benchmarking, try to come up with a _useful_ benchmark
instead.  One that allows people to make a reasoned decision about
choosing a particular piece of hardware/software.

Such as a disk benchmark (_real_ disk IO - e.g. multiple NNTP servers
with a big news hierarchy), network cards (_real_ network traffic
under realistic conditions, not a single packet on an unloaded CPU),
OS benchmark (behavior of _real_ programs - few people get a system to
run null syscalls but many get systems to run databases, graphics, FTP
servers, FP computing, interactive multiuser editing/mail reading/
programming, ...)
									++sja