Generally speaking what I am recommending is that races be allowed

    insofar as they don't interfere with operation, which will be in

    most cases.  The cases where it would interfere are primarily going

    to be related to userland blocking conditions (e.g. read() on socket

    blocks waiting for incoming data/disconnect/shutdown, or write() on

    socket blocks waiting for buffer space to become available).
    These blocking conditions can be resolved by sending a synchronous

    message to the protocol stack, just like we do now for connect() and

    numerous other operations.  The protocol stack would then be able

    to resolve any races that occured, potentially returning the message

    (unblocking the connection) immediately in the rare case of a race.
					---
    Ok, here are my suggestions:
    ISCONNECTED/ISCONNECTING:
	* These should only be set or cleared by the protocol stack,

	  except for a degenerate case in unp_connect2() (for socketpair()

	  I think).
	* While it is true that once one is set, then at least one or the

	  other will be found to be set under normal operation.  But

	  remote disconnection will still race, and that's an issue.
	* I believe that all such races can be resolved in the ssb_wait()

	  code.  Instead of setting a flag and tsleep()ing this code could

	  message the protocol stack (the same way we do for e.g. connect()).
	  The protocol stack would then resolve the races, potentially

	  returning the message immediately if .e.g. both bits were found

	  to be clear.
    CANTRCVMORE:
	* Set by both sides.  Should be ok as long as any races are resolved.

	  In the case of userland, it could set CANTRCVMORE and send an

	  async message to the protocol stack.
	  In the case of the protocol stack, it could set CANTRCVMORE and

	  then drain any wait-style messages (such as sent by soreceive()
    I think so_incomp is just used on listening sockets, right?  Protecting

    it with a spinlock is probably just fine.

...
[ message continues ]

On Saturday 09 August 2008, Matthew Dillon wrote:
I'm not sure what the protocol thread would have to do in this case. Isn't it

enough to just have it set the relevant flag in response to the message?
Aggelos

What do you picture the protocol side doing in response to this message?

Currently, the process side will be woken up when e.g. new data arrives or

the connection state changes etc. But to decide if anything changed since

the message was sent, the protocol essentially needs to know what the process

side "knew" when it sent the message. So we need to send a message which

includes a pointer to, say:
struct socksnap {

	int state;

	int rcvdata;

};
Is that what you had in mind?
Aggelos

On Saturday 09 August 2008, Matthew Dillon wrote:

[...]

I think I'm going to postpone answering to your suggestions until I start
Which will point you to

http://leaf.dragonflybsd.org/mailarchive/kernel/2008-05/msg00003.html

and

http://leaf.dragonflybsd.org/mailarchive/kernel/2008-06/msg00085.html
You can use the web interface at

http://repo.or.cz/w/dragonfly/netmp.git?a=shortlog;h=refs/heads/sockbuf

to browse the code or try
git clone git://repo.or.cz/dragonfly/netmp.git

cd netmp

git branch sockbuf origin/sockbuf

git checkout sockbuf
to get an on-disk copy. The code needs a simplification pass, but mostly

works now (known issues at http://wiki.dragonflybsd.org/index.cgi/NetMP).

That should take care of the so_rcvbuf, so_sndbuf issues.
Aggelos