Re: Network slowdown due to CFS

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Martin Michlmayr <tbm@...>
Cc: Mike Galbraith <efault@...>, Srivatsa Vaddagiri <vatsa@...>, <linux-kernel@...>, Stephen Hemminger <shemminger@...>
Date: Thursday, September 27, 2007 - 5:49 am

* Martin Michlmayr <tbm@cyrius.com> wrote:


Martin, could you check the iperf patch below instead of the yield patch 
- does it solve the iperf performance problem equally well, and does CPU 
utilization drop for you too?

	Ingo

-------------------------->
Subject: iperf: fix locking
From: Ingo Molnar <mingo@elte.hu>

fix iperf locking - it was burning CPU time while polling
unnecessarily, instead of using the proper wait primitives.

Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 compat/Thread.c |    3 ---
 src/Reporter.c  |   13 +++++++++----
 src/main.cpp    |    2 ++
 3 files changed, 11 insertions(+), 7 deletions(-)

Index: iperf-2.0.2/compat/Thread.c
===================================================================
--- iperf-2.0.2.orig/compat/Thread.c
+++ iperf-2.0.2/compat/Thread.c
@@ -405,9 +405,6 @@ int thread_numuserthreads( void ) {
 void thread_rest ( void ) {
 #if defined( HAVE_THREAD )
 #if defined( HAVE_POSIX_THREAD )
-    // TODO add checks for sched_yield or pthread_yield and call that
-    // if available
-    usleep( 0 );
 #else // Win32
     SwitchToThread( );
 #endif
Index: iperf-2.0.2/src/Reporter.c
===================================================================
--- iperf-2.0.2.orig/src/Reporter.c
+++ iperf-2.0.2/src/Reporter.c
@@ -111,6 +111,7 @@ report_statistics multiple_reports[kRepo
 char buffer[64]; // Buffer for printing
 ReportHeader *ReportRoot = NULL;
 extern Condition ReportCond;
+extern Condition ReportDoneCond;
 int reporter_process_report ( ReportHeader *report );
 void process_report ( ReportHeader *report );
 int reporter_handle_packet( ReportHeader *report );
@@ -338,7 +339,7 @@ void ReportPacket( ReportHeader* agent, 
             // item
             while ( index == 0 ) {
                 Condition_Signal( &ReportCond );
-                thread_rest();
+                Condition_Wait( &ReportDoneCond );
                 index = agent->reporterindex;
             }
             agent->agentindex = 0;
@@ -346,7 +347,7 @@ void ReportPacket( ReportHeader* agent, 
         // Need to make sure that reporter is not about to be "lapped"
         while ( index - 1 == agent->agentindex ) {
             Condition_Signal( &ReportCond );
-            thread_rest();
+            Condition_Wait( &ReportDoneCond );
             index = agent->reporterindex;
         }
         
@@ -553,6 +554,7 @@ void reporter_spawn( thread_Settings *th
         }
         Condition_Unlock ( ReportCond );
 
+again:
         if ( ReportRoot != NULL ) {
             ReportHeader *temp = ReportRoot;
             //Condition_Unlock ( ReportCond );
@@ -575,9 +577,12 @@ void reporter_spawn( thread_Settings *th
                 // finished with report so free it
                 free( temp );
                 Condition_Unlock ( ReportCond );
+            	Condition_Signal( &ReportDoneCond );
+		if (ReportRoot)
+			goto again;
             }
-            // yield control of CPU is another thread is waiting
-            thread_rest();
+            Condition_Signal( &ReportDoneCond );
+            usleep(10000);
         } else {
             //Condition_Unlock ( ReportCond );
         }
Index: iperf-2.0.2/src/main.cpp
===================================================================
--- iperf-2.0.2.orig/src/main.cpp
+++ iperf-2.0.2/src/main.cpp
@@ -96,6 +96,7 @@ extern "C" {
     // records being accessed in a report and also to
     // serialize modification of the report list
     Condition ReportCond;
+    Condition ReportDoneCond;
 }
 
 // global variables only accessed within this file
@@ -141,6 +142,7 @@ int main( int argc, char **argv ) {
 
     // Initialize global mutexes and conditions
     Condition_Initialize ( &ReportCond );
+    Condition_Initialize ( &ReportDoneCond );
     Mutex_Initialize( &groupCond );
     Mutex_Initialize( &clients_mutex );
 

-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Network slowdown due to CFS, Martin Michlmayr, (Wed Sep 26, 4:52 am)
Re: Network slowdown due to CFS, Mike Galbraith, (Wed Sep 26, 6:23 am)
Re: Network slowdown due to CFS, Martin Michlmayr, (Wed Sep 26, 6:48 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Sep 26, 7:21 am)
Re: Network slowdown due to CFS, Martin Michlmayr, (Wed Sep 26, 7:29 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Thu Sep 27, 5:49 am)
Re: Network slowdown due to CFS, Martin Michlmayr, (Thu Sep 27, 6:54 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Thu Sep 27, 6:56 am)
Re: Network slowdown due to CFS, Martin Michlmayr, (Thu Sep 27, 7:12 am)
RE: Network slowdown due to CFS, David Schwartz, (Wed Sep 26, 8:00 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Sep 26, 9:31 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Thu Sep 27, 5:30 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Thu Sep 27, 5:46 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Thu Sep 27, 8:27 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Thu Sep 27, 9:31 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Thu Sep 27, 10:42 am)
Re: Network slowdown due to CFS, Nick Piggin, (Fri Sep 28, 2:10 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Mon Oct 1, 4:43 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Tue Oct 2, 5:26 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Mon Oct 1, 12:25 pm)
Re: Network slowdown due to CFS, Jarek Poplawski, (Tue Oct 2, 5:03 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Tue Oct 2, 9:39 am)
Re: Network slowdown due to CFS, Chris Friesen, (Mon Oct 1, 12:55 pm)
Re: Network slowdown due to CFS, Ingo Molnar, (Mon Oct 1, 1:09 pm)
Re: Network slowdown due to CFS, Chris Friesen, (Mon Oct 1, 1:45 pm)
Re: iperf yield usage, Ingo Molnar, (Mon Oct 1, 3:09 pm)
RE: Network slowdown due to CFS, David Schwartz, (Mon Oct 1, 12:49 pm)
RE: Network slowdown due to CFS, Rusty Russell, (Wed Oct 3, 8:31 pm)
Re: Network slowdown due to CFS, Helge Hafting, (Wed Oct 3, 7:31 am)
Re: Network slowdown due to CFS, Arjan van de Ven, (Mon Oct 1, 3:53 pm)
RE: Network slowdown due to CFS, David Schwartz, (Mon Oct 1, 6:17 pm)
Re: Network slowdown due to CFS, Arjan van de Ven, (Mon Oct 1, 6:35 pm)
RE: Network slowdown due to CFS, David Schwartz, (Mon Oct 1, 6:44 pm)
Re: Network slowdown due to CFS, Arjan van de Ven, (Mon Oct 1, 6:55 pm)
RE: Network slowdown due to CFS, David Schwartz, (Tue Oct 2, 11:37 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Wed Oct 3, 3:15 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Mon Oct 1, 1:31 pm)
RE: Network slowdown due to CFS, David Schwartz, (Mon Oct 1, 2:23 pm)
Re: yield API, Ingo Molnar, (Tue Oct 2, 2:46 am)
Re: yield API, Jesper Juhl, (Wed Dec 12, 6:39 pm)
Re: yield API, Kyle Moffett, (Thu Dec 13, 12:43 am)
RE: yield API, David Schwartz, (Thu Dec 13, 4:10 pm)
Re: yield API, Eric St-Laurent, (Tue Oct 2, 5:57 pm)
Re: yield API, linux-os (Dick Johnson), (Tue Oct 2, 7:50 am)
Re: yield API, Douglas McNaught, (Tue Oct 2, 11:24 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Tue Oct 2, 2:26 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Tue Oct 2, 2:08 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Tue Oct 2, 2:06 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Wed Oct 3, 4:02 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Oct 3, 4:16 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Wed Oct 3, 4:56 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Oct 3, 5:10 am)
Re: Network slowdown due to CFS, Casey Dahlin, (Thu Oct 4, 1:33 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Wed Oct 3, 5:50 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Sun Oct 7, 3:18 am)
Re: Network slowdown due to CFS, Dmitry Adamushko, (Wed Oct 3, 6:55 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Wed Oct 3, 7:40 am)
Re: yield, Ingo Molnar, (Wed Oct 3, 7:56 am)
Re: yield, Jarek Poplawski, (Wed Oct 3, 8:16 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Oct 3, 7:22 am)
Re: Network slowdown due to CFS, Dmitry Adamushko, (Wed Oct 3, 6:58 am)
Re: Network slowdown due to CFS, Jarek Poplawski, (Wed Oct 3, 7:20 am)
Re: Network slowdown due to CFS, Andi Kleen, (Tue Oct 2, 2:47 am)
Re: Network slowdown due to CFS, Stephen Hemminger, (Wed Sep 26, 11:46 am)
Re: Network slowdown due to CFS, Stephen Hemminger, (Wed Sep 26, 11:40 am)
Re: Network slowdown due to CFS, Mike Galbraith, (Wed Sep 26, 6:20 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Sep 26, 5:34 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Sep 26, 5:47 am)
Re: Network slowdown due to CFS, Martin Michlmayr, (Wed Sep 26, 6:08 am)
Re: Network slowdown due to CFS, Ingo Molnar, (Wed Sep 26, 6:18 am)