Return to BSD News archive
Path: sserve!newshost.anu.edu.au!munnari.oz.au!constellation!osuunx.ucc.okstate.edu!moe.ksu.ksu.edu!vixen.cso.uiuc.edu!uwm.edu!math.ohio-state.edu!darwin.sura.net!europa.eng.gtefsd.com!usenet From: niemidc@oasis.gtefsd.com (David C. Niemi) Newsgroups: comp.os.386bsd.development Subject: Re: Compressing file system ? Date: 6 Aug 1993 14:49:55 GMT Organization: GTE Federal Systems Division Lines: 71 Distribution: world Message-ID: <23tr2j$3tt@europa.eng.gtefsd.com> References: <23rh55$ct@encap.Hanse.DE> Reply-To: niemidc@oasis.gtefsd.com NNTP-Posting-Host: hengist.lab.oasis.gtegsc.com In article ct@encap.Hanse.DE, jan@encap.Hanse.DE (Jan-Oliver Neumann) writes: >Hello folks. > >I've installed DOS 6.0 (yuck!) for a friend yesterday and I was >quite impressed how little the overhead for the compressed >disks was. I have DR-DOS 6 with SuperStor, and it too is very fast on compressed partitions (not noticeably slower than a normal one, or at least not on reads). What's more, it hasn't even trashed any of my files in 6 months of use 8) >I mean: CPU performace has increased dramatically in the past >years, but disk-speed didn't grow in that manner. > >Let's imagine a file system that compresses disk blocks that >have not been used recently with a fast algorithm and blocks >that haven't been used for a long time with a slow but effective >algorithm. I think the overhead for reading disk blocks should >be very little as heavily used blocks are already in the buffer >cache. The compressing of disk-blocks could be done asynchrously >when the system is idle or the load is very low. > >Those fstype would be ideal for partitions with news of other >text-files. > >But I also see the technical problems: disk blocks must be large >for the compressing to be effective. But large blocks also force >the fragement size to be larger. And this isn't very good for >news-partitions. Also, what about seeks ? I think bmap() can't find >out which block to read in, 'cause it doesn't know where byte 'n' >can be found. Then the file must be read in (and decompressed) >linearly to find the block. And what about the inodes ? I think >the disk-inode must contain the compressed file size and the inode >that are in-core must contain both. This implies a lot of decompressing. I think the way you'd want to do it is to implement something like the BSD FFS using small blocks underneath (perhaps 4K blocks and 1K frags) and bigger blocks from the outside world's perspective (perhaps 8K and 2K). Then you compress pages from files and fit them into the smaller blocks/frags as you write them. I think you want to keep directory structures uncompressed, and you want to compress EVERY file that goes onto disk (be consistent) on the fly. I'd think you could keep fragmentation manageable by using the FFS; and by compressing in pages, you can still do random seeks through the file and change a little data in the middle without having to uncompress/recompress the whole file. I think it should be a separate, optional file system type, rather than building it into ext2fs or something. As an alternative, there could be a new type of executable file that incorporates compression, and the loader would uncompress it as it executed it. This would work well for less-seldom-used, non-persistent executables. >Has somebody already thought about such a file-system ? This are only >some thoughts I made while waiting for the compressing of a 240 MB >drive. :) > >Jan-Oliver >-- >Jan-Oliver Neumann <jan@encap.Hanse.DE> >Hanse Networking e.V. Hamburg, FRG. Mail to vorstand@Hanse.DE for information. --- David C. Niemi: David.Niemi@oasis.gtegsc.com