Return to BSD News archive
#! rnews 5135 bsd
Newsgroups: comp.unix.bsd.bsdi.misc
Path: euryale.cc.adfa.oz.au!newshost.anu.edu.au!harbinger.cc.monash.edu.au!msunews!uwm.edu!chi-news.cic.net!newsfeed.internetmci.com!primus.ac.net!news.cais.net!news.fc.net!digdug.pswtech.com!psa.pencom.com!kirchhof.com!zilker.net!icus!wiz.com!marc
From: marc@wiz.com (Marc Wiz)
Subject: Re: More than 256 open files with select() ?
Organization: Wizywyg Software
Message-ID: <DM650r.5An@wiz.com>
References: <4en03h$qcf@news3.cts.com>
Date: Fri, 2 Feb 1996 21:58:03 GMT
Lines: 107
In article <4en03h$qcf@news3.cts.com>, Jay Curtis <jcurtis@cts.com> wrote:
>Okay, I've gone through the mailing list archives, read the faq and
>the Lizard book. There are lots of references to using more than 256
>open file handles per process with select. None of them has helped me
>get this to work.
I knew there was a reason I kept this stuff around! :-) Anyway here
is a message from Chris Torek that was on the bsdi-users mailing list.
Granted it's back from 1994 but I don't see why it should not apply.
From redstone.interpath.net!bsdi-users-request@cs.utexas.edu Mon May 9 16:39:43 1994
Received: from cs.utexas.edu (uucp@localhost) by frshaire.wiz.com (8.6.5/8.6.5) with UUCP id QAA09209 for marc@wiz.com; Mon, 9 May 1994 16:26:14 -0500
Received: from cs.utexas.edu by im4u.cs.utexas.edu (5.64/1.44/uucp) with SMTP
id AA11310; Mon, 9 May 94 15:28:50 -0500
Received: from mail-hub.interpath.net (redstone.interpath.net [199.72.1.7]) by cs.utexas.edu (8.6.9/8.6.9) with ESMTP id PAA06337 for <marc@wiz.com>; Mon, 9 May 1994 15:28:25 -0500
Received: by mail-hub.interpath.net (5.65/interpath_01)
id AA08490; Mon, 9 May 94 15:54:27 -0400
Resent-Date: Mon, 09 May 1994 13:53:42 -0600
Resent-Message-Id: <9405091954.AA08490@mail-hub.interpath.net>
Message-Id: <199405091953.PAA14873@BSDI.COM>
To: tuchman@vidda.cso.uiuc.edu (Allan Tuchman)
Cc: bsdi-users@BSDI.COM
Subject: Re: Number of open files per process
In-Reply-To: Your message of "Mon, 09 May 1994 11:57:27 MDT."
<9405091657.AA15698@vidda.cso.uiuc.edu>
Date: Mon, 09 May 1994 13:53:42 -0600
From: Chris Torek <torek@BSDI.COM>
Resent-From: bsdi-users@redstone.interpath.net
X-Mailing-List: <bsdi-users@mail-hub.interpath.net> archive/latest/1168
X-Loop: bsdi-users@mail-hub.interpath.net
Precedence: list
Resent-Sender: bsdi-users-request@redstone.interpath.net
Status: OR
>The limit of open files per process (OPEN_MAX) specified in
><sys/syslimits.h> is 64.
This is the default; it can be raised.
>But FD_SETSIZE in <sys/types.h> is 256. (Why do you need 256
>bits for 64 files?)
256 was chosen as a compromise between efficiency and expandability.
You can define FD_SETSIZE yourself before including <sys/types.h> to
change the maximum number of descriptors to which an `fd_set' can
refer. This affects only the select() call---all other file system
calls take integer file descriptors.
>Do these numbers reflect a hard limit in BSD/386, a static table in
>the kernel, a per-process table size, or what? Most important -- are
>they configurable?
They are indeed configurable. OPEN_MAX can be raised by anyone simply
by using the C-shell's built-in `limit' command.
>I'd like to increase both of these number to closer to 1024, though 512
>would probably be enough. Is this possible without major changes?
If you use select() in a process that has more than 256 files open,
you will need to recompile your kernel's sys_generic.c with a larger
FD_SETSIZE to avoid a crash (select() is not quite careful enough
about very large descriptors). Also, be aware that there is a
system-wide limit on the total number of open files; this is in the
variable `maxfiles' in your kernel's param.c (when you configure a
specific kernel for yourself, this copies /sys/conf/param.c to, e.g.,
/sys/compile/FOO/param.c). maxfiles defaults to:
int maxfiles = 3 * (NPROC + MAXUSERS) + 80;
where NPROC defaults to:
#define NPROC (20 + 16 * MAXUSERS)
MAXUSERS is simply the `maxusers' value in your configuration.
In other words, we have 3 * (20 + 16m + m) + 80 = 3*20 + 3*17m + 80 =
60 + 51m + 80 = 140 + 51m (where m is maxusers). Since I have m = 10
in my kernel configuration, I should have maxfiles = 140 + 510 = 650:
# gdb /bsd /dev/kmem
...
(gdb) p/d maxfiles
$1 = 650
(gdb)
and hence I cannot open more than 650 files in my kernel.
If you want one particular process to open many files, you should
raise maxfiles, either by configuring a kernel with a larger maxusers
(this will increase numerous table sizes; see param.c), or by changing
your configuration's copy of param.c to set maxfiles higher. (You can
also use gdb to rewrite `maxfiles' in an already-compiled kernel, but
this change will be lost on the next recompilation, so I would not
recommend this method. For test purposes, you can even raise maxfiles
on a running kernel; this will, of course, be lost across a reboot.)
--
In-Real-Life: Chris Torek, Berkeley Software Design Inc (+1 510 549 1145)
Berkeley, CA Domain: torek@bsdi.com
End of Chris' message. I hope this helps.
Marc
--
Marc
marc@wiz.com
Yes, that really is my last name.