Return to BSD News archive
Path: euryale.cc.adfa.oz.au!newshost.anu.edu.au!harbinger.cc.monash.edu.au!munnari.OZ.AU!spool.mu.edu!howland.reston.ans.net!newsfeed.internetmci.com!panix!not-for-mail
From: tls@panix.com (Thor Lancelot Simon)
Newsgroups: comp.os.linux.networking,comp.unix.bsd.netbsd.misc,comp.unix.bsd.freebsd.misc
Subject: Re: TCP latency
Date: 15 Jul 1996 04:32:50 -0400
Organization: Panix
Lines: 68
Message-ID: <4scvni$9ab@panix2.panix.com>
References: <4paedl$4bm@engnews2.eng.sun.com> <31E995B6.573A3461@lambert.org> <4schqh$ssa@panix2.panix.com> <31E9E122.190E8C64@lambert.org>
Reply-To: tls@rek.tjls.com
NNTP-Posting-Host: panix2.panix.com
Xref: euryale.cc.adfa.oz.au comp.os.linux.networking:45236 comp.unix.bsd.netbsd.misc:4050 comp.unix.bsd.freebsd.misc:23589
In article <31E9E122.190E8C64@lambert.org>,
Terry Lambert <terry@lambert.org> wrote:
>Thor Lancelot Simon wrote:
>] >You would map the buffer into both the source and target
>] >applications address spaces, and avoid the user->kernel->user
>] >copies.
>]
>] If you want to do that, don't you want to just change
>] copyin/copyout to copy any unaligned portion at either end,
>] and then map instead of copying the middle? This wins for
>] everything, without requiring a change in user-level semantics.
>]
>] I was lucky enough to get to discuss this with Kirk McKusick
>] at one point a few months ago. He indicated that at least one
>] version of Mach had done exactly that on the VAX, and that it
>] turned out to be a win iff you wanted to move more than about
>] 8K. Of course, that number is so utterly hardware dependent...
>
>You could change the the copyin/copyout; however, the benefit
>to using mapping of kernel buffer is page alignment.
So, if you know you'll want this, allocate your memory with mmap().
A colleague who did a lot of work with DG/UX once told me that DG/UX had
a system call to allocate page-aligned memory, presumably so device drivers
could set up DMA directly into your address space; we liked the idea and were
mulling over trying to add such a syscall to NetBSD when we realized that
it could be done using mmap() already.
>If I have a page aligned buffer that I wanted to send out the
>wire (to go back to networking) I could:
>
> ,- page boundry
> v
> [ buffer ] <- user mapping
> [ page ] [ buffer ] <- kernel mapping
> ^ ^
> | `- write header cruft here
> copy network
> data to card starting at buffer offest minu header
> size (up to 4k).
>
>It should be obvious that using this method, you chould turn an
>NFS read request around in the kernel
>
>Using DMA devices for disk and nework I/O, and having a unified
>VM cache, the cost will be:
>
> 1 DMA from disk to memory (you have to do this anyway)
> 1 page offset mapping
> 1 copy for header data
> 1 DMA from memory to network (you have to do this anyway)
>
>This should save you several copies over FreeBSD, at least one
>more copy over Linux (which has user space NFS), and additional
>per header copies over ordinary Streams.
>
>PS: Implemented this in prototype in Mentat Streams on VMS in
> 1992 or so for the Pathworks for VMS (NetWare) server.
Cute. Still, I think my idea above is more general. Make sure that MAP_ANON
always returns page-aligned memory -- it ought to already, but I haven't
looked -- and change copyin/copyout as I suggested, and don't you get more
or less what you're describing, without changing any existing semantics?
--
Thor Lancelot Simon tls@panix.COM
"Gee, if your knee jerks any harder you're going to kick yourself in the head."
-- Barry Sherman