On Mon, Oct 01, 2007 at 08:50:50PM -0700, David Miller wrote:
quoted text
> From: lm@bitmover.com (Larry McVoy)
> Date: Mon, 1 Oct 2007 19:20:59 -0700
> 
> > A short summary is "can someone please post a test program that sources
> > and sinks data at the wire speed?"  because apparently I'm too old and
> > clueless to write such a thing.
> 
> You're not showing us your test program so there is no way we
> can help you out.

Attached.  Drop it into an lmbench tree and build it.

quoted text
> My initial inclination, even without that critical information,
> is to ask whether you are setting any socket options in way?

The only one I was playing with was SO_RCVBUF/SO_SNDBUF and I tried
disabling that and I tried playing with the read/write size.  Didn't
help.

quoted text
> In particular, SO_RCVLOWAT can have a large effect here, if you're
> setting it to something, that would explain why dd is doing better.  A
> lot of people link to "helper libraries" with interfaces to setup
> sockets with all sorts of socket option settings by default, try not
> using such things if possible.

Agreed.  That was my first thought as well, I must have been doing 
something that messed up the defaults.  But you did get the strace
output, there wasn't anything weird there.

quoted text
> You also shouldn't dork at all with the receive and send buffer sizes.
> They are adjusted dynamically by the kernel as the window grows.  But
> if you set them to specific values, this dynamic logic is turned off.

Yeah, dorking with those is left over from the bad old days of '95
when lmbench was first shipped.  But I turned that all off and no
difference.

So feel free to show me where I'm an idiot in the code, but if you
can't, then what would rock would be a little send.c / recv.c that
demonstrated filling the pipe.
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com