Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Ingo Molnar
Date: Thursday, May 29, 2008 - 1:45 am

* Ingo Molnar <mingo@elte.hu> wrote:


update: in the past 5 days of -tip testing i've gathered about 10 
randconfig kernel configs that all produced such failures.

Since the bug itself is very elusive (it takes up to 50 boot + 
kernel-rebuild-via-distccc iterations to trigger) bisection was still 
not an option - but with 10 configs statistical analysis of the configs 
is now possible.

I made a histogram of all kernel options present in those configs, and 
one networking related kernel option stood out:

      5 CONFIG_TCP_CONG_ADVANCED=y
      6 CONFIG_INET_TCP_DIAG=y
      6 CONFIG_TCP_MD5SIG=y
      9 CONFIG_TCP_CONG_CUBIC=y

that code is called in the bootlogs:


the likelyhood of CONFIG_TCP_CONG_CUBIC=y being enabled in my randconfig 
runs is 75%. The likelyhood of CONFIG_TCP_CONG_CUBIC=y being enabled in 
10 configs in a row is 0.75^10, or 5.6%. So statistical analysis can say 
it with a 95% confidence that the presence of this option correlates to 
the hung sockets.

i have started testing this theory now, via the patch below, which turns 
off TCP_CONG_CUBIC. It will take about 50 bootups on the affected 
testsystems to confirm. (it will take a couple of hours today as not all 
testsystems show these hung socket symptoms)

distributions enable TCP_CONG_CUBIC by default:

  $ grep CUBIC /boot/config-2.6.24.7-92.fc8
  CONFIG_TCP_CONG_CUBIC=y
  CONFIG_DEFAULT_CUBIC=y

which would explain why Arjan and Peter triggered similar hangs as well.

	Ingo

---------------------->
Subject: qa: no TCP_CONG_CUBIC
From: Ingo Molnar <mingo@elte.hu>
Date: Thu May 29 09:45:51 CEST 2008

---
 net/ipv4/Kconfig |    4 ++++
 1 file changed, 4 insertions(+)

Index: tip/net/ipv4/Kconfig
===================================================================
--- tip.orig/net/ipv4/Kconfig
+++ tip/net/ipv4/Kconfig
@@ -454,6 +454,8 @@ config TCP_CONG_BIC
 config TCP_CONG_CUBIC
 	tristate "CUBIC TCP"
 	default y
+	depends on BROKEN_BOOT_ALLOWED
+	select BROKEN_BOOT
 	---help---
 	This is version 2.0 of BIC-TCP which uses a cubic growth function
 	among other techniques.
@@ -608,6 +610,8 @@ endif
 config TCP_CONG_CUBIC
 	tristate
 	depends on !TCP_CONG_ADVANCED
+	depends on BROKEN_BOOT_ALLOWED
+	select BROKEN_BOOT
 	default y
 
 config DEFAULT_TCP_CONG
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[bug] stuck localhost TCP connections, v2.6.26-rc3+, Ingo Molnar, (Mon May 26, 4:56 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Mon May 26, 6:28 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Mon May 26, 7:43 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Mon May 26, 7:58 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Arjan van de Ven, (Mon May 26, 9:24 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Mon May 26, 9:32 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Mon May 26, 10:08 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ingo Molnar, (Mon May 26, 11:12 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Mon May 26, 2:20 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Peter Zijlstra, (Wed May 28, 2:27 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ingo Molnar, (Thu May 29, 1:45 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Thu May 29, 4:14 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Evgeniy Polyakov, (Thu May 29, 6:05 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Thu May 29, 6:48 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ingo Molnar, (Fri May 30, 11:18 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Fri May 30, 2:12 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ingo Molnar, (Fri May 30, 11:09 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 4:46 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 5:18 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 5:58 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Sat May 31, 7:25 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 9:09 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 10:22 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Sat May 31, 10:58 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 11:37 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Sat May 31, 1:25 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 2:39 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Sat May 31, 2:45 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Patrick McManus, (Sat May 31, 3:46 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Sat May 31, 10:51 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Eric Dumazet, (Sat May 31, 11:04 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Tue Jun 3, 5:10 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Wed Jun 4, 4:14 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Wed Jun 4, 7:00 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Wed Jun 4, 8:09 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Fri Jun 6, 2:32 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Mon Jun 9, 12:24 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Tue Jun 10, 4:26 pm)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Ilpo Järvinen, (Wed Jun 11, 6:39 am)
Re: [bug] stuck localhost TCP connections, v2.6.26-rc3+, Håkon Løvdal, (Wed Jun 18, 5:30 pm)