*BSD News Article 39282


Return to BSD News archive

Xref: sserve comp.unix.aix:48686 comp.unix.bsd:15531 comp.unix.pc-clone.32bit:7703 comp.unix.solaris:28152 comp.unix.unixware:15059
Path: sserve!newshost.anu.edu.au!harbinger.cc.monash.edu.au!msunews!uwm.edu!cs.utexas.edu!swrinde!sgiblab!sgigate.sgi.com!fido.asd.sgi.com!slovax!lm
From: lm@slovax.engr.sgi.com (Larry McVoy)
Newsgroups: comp.unix.aix,comp.unix.bsd,comp.unix.pc-clone.32bit,comp.unix.solaris,comp.unix.unixware
Subject: Re: Unix for PC
Date: 10 Dec 1994 22:01:28 GMT
Organization: Silicon Graphics Inc., Mountain View, CA
Lines: 61
Distribution: inet
Message-ID: <3cd8fo$ns5@fido.asd.sgi.com>
References: <199411210319.TAA18133@nic.cerf.net> <D0E32G.3x8@news.cern.ch> <MICHAELV.94Dec10124723@MindBender.HeadCandy.com>
Reply-To: lm@slovax.engr.sgi.com
NNTP-Posting-Host: slovax.engr.sgi.com
X-Newsreader: TIN [version 1.2 PL2]

Michael L. VanLoon (michaelv@MindBender.HeadCandy.com) wrote:
: In article <3c81c7$h1o@fido.asd.sgi.com> lm@fubar (Larry McVoy) writes:

:    Nate Williams (nate@bsd.coe.montana.edu) wrote:
:    : C'mon Dan.  Commercial OS software testing is completely different than
:    : free software testing in general.  The reason Linux and FreeBSD have

:    I hate to burst your bubble, but I worked at Sun in the systems group for
:    a few years (and then in the server group).  They had *no* regression 
:    test other than the binaries that shipped with the OS.  Since 5.x,
:    they use the POSIX test suites but those (were) are pathetic and 
:    certainly don't cover everything.

: I've worked with a certain very large software company and it is not
: done like this at all.  They do extensive testing constantly during
: the development cycle, before anyone outside the company even sees it.

I think you are misunderstanding the point.  Certainly Sun, and every
other big or small company, will run the new release internally before
shipping it and will "test" that the binaries "work".

I think we are arguing about the definition of test and work.  "It
boots" is not the same as testing, nor is running bunch of makes and
ftps, whatever.  Certainly those are all good things but they are far
from conclusive.  If it was that easy to test the software and insure
no bugs then every company would do so, they aren't stupid.

The point is that there is very little in the way of formal regression
testing.  Even to the point of testing for old bugs.  You would think
that any company that fixes a bug would add a small test to a
regression suite that tested for that bug as a part of the fix.  Sun
doesn't.  I don't know of any major Unix vendor that does.

Most of the testing falls under one of the following catagories:

	1) run it as a server for builds and/or NFS service (the latter
	is a pretty hard thing to get people to do - internal organizations
	don't want their NFS servers going down either).

	2) Run some sort of conformance test suite such as POSIX.  This is
	painful and not done frequently.

	3) Run it on the developer's desktops.  This is the most common.

I'd love to "raise the bar" for the industry as a whole by having
someone out there describe their testing procedure that goes beyond
what I described.

I think that more testing is good.

I think that no amount of testing finds everything.

I think that people don't like hearing that their favorite OS might not
be bullet proof but that is a fact of life.

The original point was that Linux is less tested than the major vendor's
releases.  That may be true but the gap between Linux and a commercial
release is much smaller than you seem to think.
--
---
Larry McVoy			(415) 390-1804			 lm@sgi.com