This issue come from ruby language community. Below test program
hang up when only run on Linux.
% uname -mrsv
Linux 2.6.26-2-486 #1 Sat Dec 26 08:37:39 UTC 2009 i686
% ruby -rsocket -ve '
BasicSocket.do_not_reverse_lookup = true
serv = TCPServer.open("127.0.0.1", 0)
s1 = TCPSocket.open("127.0.0.1", serv.addr[1])
s2 = serv.accept
s2.close
s1.write("a") rescue p $!
s1.write("a") rescue p $!
Thread.new {
s1.write("a")
}.join'
ruby 1.9.3dev (2010-07-06 trunk 28554) [i686-linux]
#<Errno::EPIPE: Broken pipe>
[Hang Here]
FreeBSD, Solaris, Mac doesn't. because Ruby's write() method call
select() internally. and tcp_poll has a bug.
SUS defined 'ready for writing' of select() as following.
| A descriptor shall be considered ready for writing when a call to an output
| function with O_NONBLOCK clear would not block, whether or not the function
| would transfer data successfully.
That said, EPIPE situation is clearly one of 'ready for writing'.
We don't have read-side issue because tcp_poll() already has read side
shutdown care.
| if (sk->sk_shutdown & RCV_SHUTDOWN)
| mask |= POLLIN | POLLRDNORM | POLLRDHUP;
So, Let's insert same logic in write side.
- reference url
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/31065
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/31068
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
net/ipv4/tcp.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 176e11a..2497e48 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -451,7 +451,8 @@ unsigned int tcp_poll(struct file *file, struct socket *sock, poll_table *wait)
if (sk_stream_wspace(sk) >= sk_stream_min_wspace(sk))
mask |= POLLOUT | POLLWRNORM;
}
- }
+ } else
+ mask |= POLLOUT | POLLWRNORM;
if (tp->urg_data & TCP_URG_VALID)
mask |= POLLPRI;
--
1.6.5.2
--
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> How Linux should behave is defined by many things, and often it is simply defined by how we've behaved for a very long time. This is because changing behavior can often break as many applications as it can fix. Standards don't necessarily tell us how we must behave, since often is it impractical to follow their definions. And in this case here, I call into question the behavior of Ruby and the application from two perspectives: 1) Unlike all of the other conditions signalled by poll() this is one the application explicitly created and therefore knows about. If the application calls close() or shutdown() with the send flag set, IT KNOWS what is going to happen on a write() attempt. 2) Ruby and this script will have to deal with the past 13 years worth of Linux kernels. Even if I were to apply this fix now it is not going to propagate onto a user's system any time soon. Many systems would never ever get this fix. Therefore it behooves Ruby and this script to make a very reasonable change, which is to track when close() or send shutdown() calls occur and behave appropriately on a write() call. I'm therefore not applying this patch, because not only can applications handle this properly with information they already have, the change has the potential to break applications. --
[...]
This example seems to have both server (serv, s2) and client (s1) in the
same process for simplicity. The server socket (s2) is closed and the
client cannot be expected to know that. Of course the client ought to
drop the connection after the first EPIPE, but it's reasonable to expect
that this is a sticky condition just as it would be for a pipe.
Here's a similar test case in C:
#include <assert.h>
#include <signal.h>
#include <stdio.h>
#include <sys/select.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
int main(void)
{
struct sockaddr sa;
struct timeval tv;
int serv, s1, s2;
socklen_t len;
fd_set fds;
signal(SIGPIPE, SIG_IGN);
serv = socket(AF_INET, SOCK_STREAM, 0);
assert(serv >= 0);
assert(!listen(serv, 1));
len = sizeof(sa);
assert(!getsockname(serv, &sa, &len));
s1 = socket(AF_INET, SOCK_STREAM, 0);
assert(s1 >= 0);
assert(!connect(s1, &sa, len));
len = sizeof(sa);
s2 = accept(serv, &sa, &len);
assert(s2 >= 0);
close(s2);
for (;;) {
printf("write: %d\n", write(s1, "a", 1));
FD_ZERO(&fds);
FD_SET(s1, &fds);
tv.tv_sec = 1;
tv.tv_usec = 0;
printf("select: %d\n", select(s1 + 1, NULL, &fds, NULL, &tv));
}
return 0;
}
Ben.
--
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
--
Cool! Ben, I think your code is cleaner than mine. If you allow me, I hope to include this one into my patch description. Thanks. --
On Thu, 2010-08-26 at 12:55 +0900, KOSAKI Motohiro wrote: You can do that if you like, but it seems a bit long for a commit message. By the way, I was able to reproduce this bug in RHEL 3 (kernel based on 2.4.21) so it seems to have been around for a while. Ben. -- Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked. --
From: Ben Hutchings <bhutchings@solarflare.com>
It's existed since September 1998:
commit 6275af56642b5b9ed9ef15bf5d9718661c5fe79d
Author: freitag <freitag>
Date: Sat Sep 26 06:20:25 1998 +0000
Some small fixes.
- Remove second check for sk->err in poll as discussed with Linus.
- Don't signal writability when the local end has been shut down.
- Comment fixes.
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 5a09639..16d4308 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -5,7 +5,7 @@
*
* Implementation of the Transmission Control Protocol(TCP).
*
- * Version: $Id: tcp.c,v 1.123 1998-09-23 19:52:39 davem Exp $
+ * Version: $Id: tcp.c,v 1.124 1998-09-26 06:20:25 freitag Exp $
*
* Authors: Ross Biro, <bir7@leland.Stanford.Edu>
* Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG>
@@ -591,18 +591,19 @@ unsigned int tcp_poll(struct file * file, struct socket *sock, poll_table *wait)
mask |= POLLHUP;
/* Connected? */
- if ((1 << sk->state) & ~(TCPF_SYN_SENT|TCPF_SYN_RECV)) {
+ if ((1 << sk->state) & ~(TCPF_SYN_SENT|TCPF_SYN_RECV)) {
if ((tp->rcv_nxt != tp->copied_seq) &&
(tp->urg_seq != tp->copied_seq ||
tp->rcv_nxt != tp->copied_seq+1 ||
sk->urginline || !tp->urg_data))
mask |= POLLIN | POLLRDNORM;
- /* Always wake the user up when an error occurred */
- if (sock_wspace(sk) >= tcp_min_write_space(sk, tp) || sk->err) {
- mask |= POLLOUT | POLLWRNORM;
- } else {
- sk->socket->flags |= SO_NOSPACE; /* send SIGIO later */
+ if (!(sk->shutdown & SEND_SHUTDOWN)) {
+ if (sock_wspace(sk) >= tcp_min_write_space(sk, tp)) {
+ mask |= POLLOUT | POLLWRNORM;
+ } else { /* send SIGIO later */
+ sk->socket->flags |= SO_NOSPACE;
+ }
}
if (tp->urg_data & URG_VALID)
@@ -733,6 +734,9 @@ static void wait_for_tcp_memory(struct sock * sk)
lock_sock(sk);
}
+/* When all user supplied data has been queued set the PSH bit */
+#define PSH_NEEDED (seglen == 0 && ...because (almostly) nobody were using this feature? ;) --
Hi Umm?? Probably my example is not so good. That's not my point. In the example application, client and server socket is in the same process. But it's NOT generic. usually, client and server are another process. then, client can't expect when server close socket. The most big matter is, this is can't be avoided in userland. In addition, EVERY application don't want application hang up. we don't hesitate At first, I was thinking two fix plan. 1) this patch 2) adding POLLWRHUP as POLLRDHUP. However I couldn't find any regression rick in (1). then I did choice (1). So, Can you please tell us what rick you worry? My thinking is, If select(writefds) returned, an application naturally call to write. (why not? If not, why do you call select?) and write return EPIPE. every network application have EPIPE error checking. But, there is any rick. I can remake a patch as (2). Thanks. --
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Ok, you and Ben Hutchings have convinced me to reconsider. And this matches what even BSD4.4-Lite does (I checked yesterday before my initial reply), so I will apply this patch. Thanks for your patience. --
