Re: Linux Kernel Markers - performance characterization with large IO load on large-ish system

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Mathieu Desnoyers <mathieu.desnoyers@...>
Cc: <linux-kernel@...>, btrace <linux-btrace@...>, Jens Axboe <jens.axboe@...>
Date: Wednesday, September 26, 2007 - 11:28 am

Mathieu Desnoyers wrote:
Mathieu:

Here are the results from 6 different kernels (including ones with 
blktrace not configured in), with now performing 40 runs per kernel.

  o  All kernels start off with Linux 2.6.23-rc6 + 2.6.23-rc6-mm1

  o  '- bt cfg' or '+ bt cfg' means a kernel without or with blktrace 
configured respectively.

  o  '- markers' or '+ markers' means a kernel without or with the 
11-patch marker series respectively.

38 runs without blk traces being captured (dropped hi/lo value from 40 runs)

Kernel Options       Min val    Avg val    Max val    Std Dev
------------------  ---------  ---------  ---------  ---------
- markers - bt cfg  15.349127  16.169459  16.372980   0.184417
+ markers - bt cfg  15.280382  16.202398  16.409257   0.191861

- markers + bt cfg  14.464366  14.754347  16.052306   0.463665
+ markers + bt cfg  14.421765  14.644406  15.690871   0.233885

38 runs with blk traces being captured (dropped hi/lo value from 40 runs)

Kernel Options       Min val    Avg val    Max val    Std Dev
------------------  ---------  ---------  ---------  ---------
- markers + bt cfg  24.675859  28.480446  32.571484   1.713603
+ markers + bt cfg  18.713280  27.054927  31.684325   2.857186

  o  It is not at all clear why running without blk trace configured 
into the kernel runs slower than with blk trace configured in. (9.6 to 
10.6% reduction)
    
  o  The data is still not conclusive with respect to whether the marker 
patches change performance characteristics when we're not gathering 
traces. It appears
that any change in performance is minimal at worst for this test.
    
  o  The data so far still doesn't conclusively show a win in this case 
even when we are capturing traces, although, the average certainly seems 
to be in its favor.
    
One concern that I should be able to deal easily with is the choice of 
the IO scheduler being used for both the volume being used to perform 
the test on, as well as the one used for storing blk traces (when 
enabled). Right now I was using the default CFQ, when perhaps NOOP or 
DEADLINE would be a better choice. If there is enough interest in seeing 
how that changes things I could try to get some runs in later this week.

Alan D. Brunelle
Hewlett-Packard / Open Source and Linux Organization / Scalability and 
Performance Group

-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Re: Linux Kernel Markers - performance characterization with..., Alan D. Brunelle, (Tue Sep 25, 10:58 am)
Re: Linux Kernel Markers - performance characterization with..., Mathieu Desnoyers, (Tue Sep 25, 1:13 pm)
Re: Linux Kernel Markers - performance characterization with..., Alan D. Brunelle, (Wed Sep 26, 11:28 am)