*BSD News Article 19362


Return to BSD News archive

Path: sserve!newshost.anu.edu.au!munnari.oz.au!constellation!aardvark.ucs.uoknor.edu!ns1.nodak.edu!netnews.nwnet.net!ogicse!hp-cv!sdd.hp.com!math.ohio-state.edu!
 magnus.acs.ohio-state.edu!usenet.ins.cwru.edu!agate!dog.ee.lbl.gov!hellgate.utah.edu!fcom.cc.utah.edu!cs.weber.edu!terry
Newsgroups: comp.os.386bsd.development
Subject: Re: Compressing file system ?
Message-ID: <1993Aug10.222057.20600@fcom.cc.utah.edu>
From: terry@cs.weber.edu (A Wizard of Earth C)
Date: 10 Aug 93 22:20:57 GMT
Sender: news@fcom.cc.utah.edu
References: <EgN6R_K00WB5JOZpRc@andrew.cmu.edu> <1993Aug9.233341.20631@fcom.cc.utah.edu> <johnh.744999634@ficus.cs.ucla.edu>
Organization: Weber State University, Ogden, UT
Article-I.D.: fcom.1993Aug10.222057.20600
Lines: 81

In article <johnh.744999634@ficus.cs.ucla.edu> johnh@ficus.cs.ucla.edu (John Heidemann) writes:
>>Perhaps the best implementation of a compressed file scheme I have seen was
>>the one in the UCLA Ficus implementation in BSD4.4; it has many of the
>>attributes I have tagged as "desirable" (noting that the aesthetics are
>>dictated by teeny, tiny disks rather than clean kernel implementation or
>>fast access).
>
>Here I am somewhat confused.  Ficus (which is only the replicated file
>system) is not part of BSD 4.4; only the UCLA stackable layers
>interface is included.  Unfortunately, 4.4 does not include a
>compression layer, just a null layer and a user-id mapping layer.
>
>The SunOS implementation of UCLA stacking has at one time included a
>prototype compression layer.  This layer was done as part of a class
>project by Geoff Kuenning and Steve Whitney.  Unfortunately, as a
>class project, though, it lacked some of the...reliability...one might
>want before putting it in production use.  (BTW, to relative to the
>problems described above, it encoded the compression attribute in the
>file name and compressed on block boundaries.)

This is interesting; I was under the impression from UCB that the entirety
of your work was included in 4.4, and not just the stackable layering.  I
am disappointed on two fronts; the first is, that they should have taken
everything; the second is, if they weren't going to use preexisting modules,
Rosenthal makes some good points for reforming the vops interface and the
vnode itself above and beyond what your project has done.

In the process of completing my work, I have been doing name space and
attribution in a monolithic (UFS) file system as a set of extensions to
the ordinary semantics.  Like the EFS and Kuenning/Whitney work, this
encodes the selectors as part of the name information -- reducing the
set of available file names.  It's clear that there are major problems
in the UFS inode caching and the POSIX atime semantics that a vops reform
would either eliminate completely or at least ameliorate.

I was also under the impression from the paper that the compression layer
was much more real.  8-).  Compression on block boundries seems to be the
only mechanism that could easily guarantee fast random access without a
full decompression and timeout recompression of the data.  My entire world
view has become somewhat slanted toward the "generic object store" model
since realizing the directory heirarchy and file storage models were
hopelessly polluting each other in working out the directory issues.


>>I'd encourage anyone who wants to get into building file systems to do so;
>>there are very few live projects of this type out there, and I can count the
>>number of commercial file system vendors (that's all they do) on one hand.
>
>I'd have to agree with this.  I'd love to see a good Unix 
>compression layer, among other things.

Is there any way that the same UCLA stacking code could be donated to the
*BSD projects on the net (under the same restrictions given to CSRG, of
course) so that there is not a SVR4 source license requirement to see it
(given that the BSD 4.4 Lite release seems to be in danger of not happening
for the near future)?  What parts, exactly, did CSRG get?  The UFS
implementation in the paper would be a nice excuse to throw out the existing
monolithic UFS in *BSD and to start down the right road (IMO) for new file
system work.

This would provide a better base than what is currently available to the
casual developer on these systems, which is basically module loading of
file systems as functional units (per my LKM package) without the ability
to apply a layering model.

If you have seen the "Filesystem Survival Kit" for SVR4, you are already
aware that research developement in the *BSD arena is quickly shared into
more commercial areas, and to the benefit of many users.

The first candidate for this could easily be seperation of the RockRidge
extensions from the ISO9960 FS as a seperate functional layer.  This would
provide a good proof-of-concept test for seperation of directory services
from storage services.


					Regards,
					Terry Lambert
					terry@icarus.weber.edu
---
Any opinions in this posting are my own and not those of my present
or previous employers.