Return to BSD News archive
Path: sserve!newshost.anu.edu.au!munnari.oz.au!news.Hawaii.Edu!ames!saimiri.primate.wisc.edu!zaphod.mps.ohio-state.edu!cs.utexas.edu!uunet!mcsun!sun4nl!tuegate.tue.nl!terra.stack.urc.tue.nl!tricky.wft.stack.urc.tue.nl!jim
From: jim@wft.stack.urc.tue.nl (Jim Rump)
Newsgroups: comp.os.386bsd.apps
Subject: Re: fsp ported to 386bsd
Message-ID: <1op5dp$6m1@tricky.wft.stack.urc.tue.nl>
Date: 24 Mar 93 08:16:57 GMT
References: <C4CMvI.AI7@chinet.chi.il.us>
Organization: Eindhoven University of Technology, the Netherlands
Lines: 434
NNTP-Posting-Host: shark.wft.stack.urc.tue.nl
X-Newsreader: TIN [version 1.1 PL8]
Randy Suess (randy@chinet.chi.il.us) wrote:
: In article <7728@tuegate.tue.nl> jim@blade.stack.urc.tue.nl (Jim Rump) writes:
: >I ported fsp.266.1 to 386bsd. The source are available at:
: >ftp.stack.urc.tue.nl:/pub/386bsd/0.1-ports/utils/fsp/fsp.266.1.tar.z
: >
: What is fsp?
What is the purpose of FSP (V2.6):
FSP is a set of programs that implements a public-access archive
similar to an anonymous-FTP archive. It is not meant to be a
replacement for ftp; it is only meant to do what anonymous-ftp
does, but in a manner more acceptible to the provider of the
service and more friendly to the clients.
Providing anonymous-FTP service can be costly --- each active
session consumes one process slot in the OS and one stream socket
entry in the network sub-system. The servers can also run
concurrently, adding to the system load. A popular archive site
can easily be overwhelmed as a result. Some were forced to
shutdown and some impose inconvenient access restrictions.
Unlike FTP, FSP is connection-less and virtually state-less. One
server handles requests from all clients machines. Each active
client machine takes up 16-bytes in a dynamically extensible
table. Since only one server exists on a server machine at any
time, the load added to the server machine is no more than one.
In exchange for allowing site operators to keep their sites open
and do away with cumbersome access restrictions, this is what the
clients accept with FSP:
1) Lower transfer rate. The maximum rate is 1 kbyte per UDP
message round-trip time between the client and the server.
In addition to the potential for more abundant sites and more
accessible sites, this is what the clients gain with FSP:
1) Robustness. Since FSP is connectionless, flucturations in
the network will not abort a FSP transaction. Furthermore,
the 16-bytes of data for each client can be regenerated at
any point during any transaction. Thus, if the server goes
down at any point during a transaction, the transaction will
resume when the server is restarted. (like NFS)
2) Friendlier user interface. FSP does not have its own command
interpretor like FTP. Since it is connectionless, there is
no reason to carry much information from one command to the
next, and the commands can all be made into individual unix
programs. For instance, there is one program you run to list
the directory and another you run to download a file.
3) Client protection. FSP oversees a directory structure similar
to that of an anonymous-FTP. However, a directory created
via FSP transaction is owned by the client machine that issued
the creation request. The client can create and delete files
and subdirectories in that directory. In addition, the client
can enable any of the four attributes for that directory:
A) Give all other clients the permission to create files.
B) Give all other clients the permission to delete files
or subdirectories.
C) Give all other clients the permission to read files.
(this is true by default)
D) Give all other clients the permission to create sub-
directories.
Note: A subdirectory can be deleted if it is empty and the
client owns the subdirectory.
4) Server protection. FSP server does not spawn sub-programs.
It will accept only paths that are downward relative to its
designated working directory. On systems with symbolic links,
the server will follow symbolic links, but it does not follow
uplinks (".."). Clients cannot create symbolic links and
care should be taken so that other users on the server machine
cannot create symbolic links in the server's work space.
It is also fairly difficult to formuate an attack to force a
shutdown of a FSP site by actions of a rogue site. About the
only way to distrupt a FSP service is to flood the FSP site
with network packets. FSP server prevents itself from
'counter-flooding' by filtering for legitimate requests using
the following method:
A) Each request message contains a key. For each client,
server database contains the keys to be used for the
next client request and for the previous client request.
B) If the next request does not contain a key that matches
either of the two keys, it is accepted only if at least
one minute has elapsed since the last time a request
is accepted. If the key does match the old key
(retransmit by client) it is accepted if the elapse time
is greater than 3 seconds.
C) Every request message accepted is acknowledged with
one reply message. The reply message contains a new
key to used for the next request. The new key is
computed by the server with a pseudo-random number
generator.
Flooding is a ballant violation of network etiquette because
a site can be subjected to flooding attack whether it has FSP
running or not, and flooding congests every link and gateway
between the rogue client and the server. As a further measure
of protection, the server loads a table of rogue clients on
startup. The server will not respond to requests from any of
those clients.
The software set:
common_def.h This C header file contains definitions common to
both the server code and the client code.
client_def.h This C header file contains definitions for the
client code.
server_def.h This C header file contains definitions for the
server code.
udp_io.c This file contains the lowest level routines that
deal with the unix inet sockets. This file is
used by both the server code and the client code.
server_main.c Main routine and dispatch loop for the server.
server_host.c Routines for maintaining client database.
server_file.c Routines for file i/o.
server_lib.c Routines for inet socket i/o.
client_lib.c Core routines of the client library.
client_util.c Supplementry routines of the client library.
client_lock.c udp packet multiplexing mechanism.
bsd_src/ Directory containing additional sources derived
from those in public archive on uunet.uu.net. It
contains a BSD random/srandom routine, a modified
BSD globbing routine, a modified "ls" source.
fcdcmd.c These compiles into individual client utilities.
fgetcmd.c Those with a "cmd" in their name will do their
flscmd.c own globbing on their argv base on directory
fprocmd.c information obtained from the server.
frmcmd.c
frmdircmd.c
fcatcmd.c
fmkdir.c
fput.c
fver.c
fgrab.c
merge.c This file is used to call the other client programs
when program merging option is used. Rather than
compiling into individual client utility programs,
users may opt to build a single program which
contains all client funtionalities, which are to
be distinguished at run time by the name used to
activate the program. Significant space saving can
be achieved on some machines.
Compilation:
FSP has been compiled and tested by me on a SS-2 running SunOs
4.1.1, a HP-9000 running HP UNIX, a VAX-780 running 4.3-tahoe,
and a 386 box running system-V UNIX with old Excelan ethernet
interface. It has also been compiled on a variety of machines
by over hundreds users all over the world.
To compile the software, you must first successfully complete a
"make" in the bsd_src directory. You may have to change a few
files. In particular, you may have to edit "Makefile" and "tweak.h"
in bsd_src directory.
When that is done, you can edit the Makefile on the top directory
and run "make" in the top directory, or "make merge" if you want
to combine all client programs into one single file. You may wish
to read through the rest of this document first before making
changes to the Makefile.
After "make" or "make merge" finished, you may use "make install"
or "make install-merge" to install the programs. Man pages may be
installed by "make installm".
IMPORTANT NOTE: You do not need to run the fspd process if you only
want to access existing fsp archives. fspd processs is needed only
if you want to setup and archive for public access.
Server Administration:
The only things you need for setting up a FSP server is a work
directory for the service and and the FSP server itself (fspd).
fspd can run independently or it can be run under inetd. When
running independently, fspd waits for messages through a UDP
socket whoes port number is defined in the Makefile. When running
under inetd, fspd is involked as in.fspd. inetd will spawn fspd
when a message arrives for the FSP socket. The fspd process will
take over and stick around to wait on additional messages. After
it has become idle for 2 minutes, fspd will exit and return control
to inetd.
Sample setup for inetd operation:
In /etc/services file:
fsp 21/udp fspd
In /etc/inetd.conf file:
fsp dgram udp wait ftp /usr/etc/fspd in.fspd
In this sample, the same port number for ftp is used for the
fsp socket. There will not be a conflict because ftp uses
stream protocol, and fsp uses UDP protocol. The fspd program
in this example is ran under user 'ftp'.
In addition, fspd will accept these flags:
-h absolute_path Set fsp work directory. Overrides the
compiled-in default.
-p udp_port_number Set UDP port number. Overrides the
compiled-in default.
-u uid_number Assume this uid after startup. If present,
fspd will attempt a setuid() to this uid
number. It will exit if setuid() fails.
-d Turn on debug mode. The stdio files will
remain open in debugging mode.
FSPD will give info about current throughput.
-r Runs the server in restricted mode. When the
server is run in this mode, connections from
any site NOT in the .IPHOSTS files are
ignored. This is the opposite of normal
behavior, and provides a cheap way to do
priveledged access only servers.
-l [logfile] Some simple getfile/version/error logging.
Output will be sent to the given file or
to the default compile in file if no file
is specified.
-t maxthruput If specified, FSPD will limit its throughput
to maxthruput bytes per second. I recommend
20000 bytes per second if you have good links.
There is no control on uploads. The thoughput
is computed on packets sent, not recieved.
When fspd starts, it chdir to its work directory where it looks
for (and reads in if found) the file ".IPHOSTS". This file is
prepared by the FSP maintainer and indicates what action should
be taken by an FSP connect from the given host.
The format for the file is as follows.
hostmask [type] [message]
hostmask may be of the form xx.xx.xx.xx where xx is either
an integer, an integer range (specified as xx-xx) or * which
is equivalent to 0-255, or hostmask may be the fully qualified
DNS name of a host (no wildcarding is allowed in this mode
currently).
type is either N, I, or D. N means a normal host. Connections
from a normal host act exactly like connections do today. I is
an ignored host. Connections from an I host will not get any
response (not even an error) from the server. D means disabled
and in this case, the message parameter will be sent back for
every request from that host. If type is missing, it acts like
either N or I depending on the setting of the -r flag.
message is an optional parameter only used if the host type is
D. This parameter has no meaning otherwise.
If a file .OWN.XXXXXXXX, where XXXXXXXX is an 8-digit hex number,
exists in a directory in fspd's work space, the directory is owned
by the machine whoes inet number is XXXXXXXX (an integer stored
in the network byte order), where the number is printed as a
hexadecimal number. If no such file exists, the directory has
no owner. (Note, the 'dot' files are hidden from clients).
If the file .FSP_OK_DEL does not exists in a directory, only the
owner is allowed to remove items from that directory.
If the file .FSP_PRIVATE exists in a directory, only the owner
is allowed to read items from that directory.
If the file .FSP_OK_ADD does not exists in a directory, only the
owner is allowed to add items into that directory.
If the file .FSP_OK_MKDIR does not exists in a directory, only
the owner is allowed to make subdirectories into that directory.
Thus, you typically want to protect the top directory by leaving
out the .FSP_OK_DEL, .FSP_OK_ADD files, and .OWN.XXXXXXXX files
in the top directory.
If the file .README is present in a directory, FSPD will read
it and send it along to the client when the directory is entered.
Currently, a .README file can only be set up by the site maintainer.
Clients do not get to read the directory information directly.
Instead, fspd maintains a directory listing for each directory
in a cache file. If the directory is writable by fspd, or if a
writable file in it is prepared beforehand, fspd will store the
directory information in .FSP_CONTENT file in that directory.
Otherwise, it will store the information in a pair of files (with
hashed names) in a special directory specified by the variable
DEF_DIRECTORY_CACHE in Makefile. The later allows read-only
directories to be exported.
When a client requests information for a directory, the cache
file is created if it doesn't exist, and it is rebuilt if it is
out of date. The information is accessed by having the client
read the directory listing file. Care is taken so that the client
will not get corrupted entries when the directory is changed while
the listing is being read.
Files being uploaded are first written to a temporary file in the
work directory: .TXXXXXXXXYYYY where XXXXXXXX is the inet number
of the client, and YYYY is the port number of the client program.
When upload is compelete, the file is moved into the intended
location.
Sending it an 'alarm' signal will cause fspd to dump its current
client database into the file .HTAB_DUMP in the work directory.
This can be useful for debugging and for catching rogue clients.
Client utilities:
All inter-command states are kept in these three shell environment
variables.
FSP_PORT Port number of the fspd you wish to contact.
FSP_HOST Host name or number of the fspd.
FSP_DIR Your current working directory in the archive.
When multiple client utilities are run at the same time on the
same client machine, packet multiplexing mechanisms can be used
to enable concurrent access to the same fsp database. If none
of the mechanisms are selected at compile time, FSP_LOCALPORT
can be used to ensure that only once client utility can run at
any time. In this case, FSP_LOCALPORT can be set to any port
number not current used on the client machine.
FSP_TRACE can be set if you want status reports be printed while
files are being transferred. FSP_DELAY variable can be used to
set the retransmit interval for client utilities (in thousandth
of a second). The retransmit rate is adjusted in an exponential
manner, until the retry rate reaches 5 minutes per retry.
FSP_BUF_SIZE can be set to a positive number less than or equal
to 1024. When set, it determines the size of data to be send for
each request during file and directory information transfer. The
default is 1024. Some sites are connected via links that cannot
transmit buffers containing 1024 bytes of data in addition to the
header information. Setting FSP_BUF_SIZE to a lower value will
allow these sites to access fsp archives.
A typical setup looks like this:
setenv FSP_PORT 21
setenv FSP_HOST 131.215.131.97
setenv FSP_DIR /
setenv FSP_TRACE
setenv FSP_DELAY 3000
setenv FSP_BUF_SIZE 1024
(All examples will be in csh. However, it is assumed that similar
things can be done with other shells)
For commands that do globbing using remote directory info, normal
shell globbing needs to be turned off. In csh, it can be done
with a set of aliases:
alias fcd setenv FSP_DIR \`\(set noglob\; exec fcdcmd \!\*\)\`
alias fls \(set noglob\; exec flscmd \!\*\)
alias fget \(set noglob\; exec fgetcmd \!\*\)
alias fgrab \(set noglob\; exec fgrabcmd \!\*\)
alias fcat \(set noglob\; exec fcatcmd \!\*\)
alias frm \(set noglob\; exec frmcmd \!\*\)
alias frmdir \(set noglob\; exec frmdircmd \!\*\)
alias fpro \(set noglob\; exec fprocmd \!\*\)
In addtion, these alias are useful:
alias fpwd echo \$FSP_DIR on \$FSP_HOST port \$FSP_PORT
alias fsethost setenv FSP_DIR \/\; setenv FSP_HOST \!\:1\; \
setenv FSP_PORT \!\:2
The second one is a user contribution. It allows one to set
the FSP_HOST, FSP_PORT, and to initialize FSP_DIR in one command.
Commands:
fver display server's version number.
fcd change current remote directory, like cd.
fls list directory. works like ls.
fget get the named files.
fgrab get the named file and delete it from remote directory.
fput put the named files.
fcat get the named files and send them to stdout.
fmkdir make named directories.
frm delete named files.
frmdir delete named directories.
fpro no arg: display directory protection modes.
+c: give others permission to create new items.
-c: deny others permission to create new items.
+d: give others permission to delete old items.
-d: deny others permission to delete old items.
***********************************************************************
This is a free software. Be creative; make your own macros and tools
and let me know of any bugs and suggestions.
A mailing list for the discussion of the FSP software is now available
(started Oct 2, 1992.) To get on the list, send an email stating
that you want to be on the FSP list to the following address:
listmaster@Germany.EU.net
Articles to be distributed to the subscribers should be sent to the
following email address:
fsp-discussion@Germany.EU.net
Jim.
PS: Many sites in the world use it already.