cc:ing as I don't know if I can actually post to tech@ 

In my own tests, when I got apalling speeds like that I discovered that the
remote connection had timestamps turned off.

I'm not sure why you're trying to both raise the starting point as well as
the increment speed.  As the normal cap is at 256k anyway.  So it can't
increment anywhere.

What I've been running with for a while is starting at 48k and raising by
32k increments.  Which doesn't appear to be too bad internationally, and
isn't quite as bad when timestamps aren't available.

I've also been running with the maximum boosted.

It's only recently I looked at the sender side stuff.  And I struggled to
understand quite how it worked.

To my mind having the default raised to 32768 at least makes sense.  I want
to look into things further myself, but I haven't got a seperate development
environment setup yet.  

What I've been thinking about is that sometimes you want to increase the
window size faster, and smoetimes slower.  Depending on available bandwidth.

But I never thought about any way for that.  Like if I know a connection can
do 30 megabit internationally with 200 msec ping.  Then the window would need
to be 6 megabits to fill the link if it was a single connection.  But in the
real world often there'll be 20+ connections so then window size would not
want to be nearly that big.

That said, you realise you can still set the socket buffer size in an
application - like squid, and relayd both support built in hard coding of
socket buffer size.

My current pet peeve is that if you bounce a connection that came in with
timestamps turned off, then you're initiating another connection with
timestamps on then you can end up with one 1448 data packet, then one 12 byte
data packet in quick sucession.

Ben.

On Wed, Dec 01, 2010 at 08:45:35PM -0500, Ted Unangst wrote:
quoted text
> here is the patch i have ended up using since the removal of the tcp 
> sysctls.  if something doesn't change, 4.9 will be an embarrassingly bad 
> regression in network performance.  at least with prevous releases, 
> bumping the recvspace was an available workaround to sucky performance, 
> but now the speedometer is permanently fixed at "balls".
> 
> i have tried fiddling with the tp->rfbuf_cnt > so->so_rcv.sb_hiwat / 8 * 7 
> formula to no avail.
> 
> my test link is ~6Mbit with ping times varying between 100 and 500ms.  so 
> basically, i get my piddly 16K, tick tock, tick tock, and then the counter 
> is reset to 0 again.  i can never get more than about 100K/s speeds, even 
> after *hours* of downloading over the same connection.  performance 
> improves to close to 600K/s with patch, which is about as good as it gets.
> 
> i know we aren't going to just crank the recvbuf to the max, so the 
> patch is really just for show, but at the very least, please bring back 
> some level of user control over recvbuf size.
> 
> [the bump to autorcvbuf_inc doesn't do much, i know, that's left over from 
> trying recvbuf smaller than the max].
> 
> Index: tcp_usrreq.c
> ===================================================================
> RCS file: /home/tedu/cvs/src/sys/netinet/tcp_usrreq.c,v
> retrieving revision 1.105
> diff -u -r1.105 tcp_usrreq.c
> --- tcp_usrreq.c	10 Oct 2010 22:02:50 -0000	1.105
> +++ tcp_usrreq.c	2 Dec 2010 01:18:28 -0000
> @@ -112,14 +112,14 @@
>  extern	struct baddynamicports baddynamicports;
>  
>  #ifndef TCP_SENDSPACE
> -#define	TCP_SENDSPACE	1024*16
> +#define	TCP_SENDSPACE	1024*64
>  #endif
>  u_int	tcp_sendspace = TCP_SENDSPACE;
>  #ifndef TCP_RECVSPACE
> -#define	TCP_RECVSPACE	1024*16
> +#define	TCP_RECVSPACE	1024*256
>  #endif
>  u_int	tcp_recvspace = TCP_RECVSPACE;
> -u_int	tcp_autorcvbuf_inc = 16 * 1024;
> +u_int	tcp_autorcvbuf_inc = 64 * 1024;
>  
>  int *tcpctl_vars[TCPCTL_MAXID] = TCPCTL_VARS;