Re: [RFC PATCH 1/2] Marker probes in futex.c

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Mathieu Desnoyers
Date: Tuesday, April 15, 2008 - 6:47 am

* Ingo Molnar (mingo@elte.hu) wrote:

I have not seen any counter argument for the in-depth analysis of the
instruction cache impact of the optimized markers I've done. Arguing
that the markers are "bloated" based only on "size kernel/sched.o"
output is a bit misleading.


You will probably be interested in the following paper, which explains
various situations in which using a tracer has solved real problems at
Google, IBM, Autodesk, which are Linux users running large clusters or
Linux systems with soft RT constraints.

Linux Kernel Debugging on Google-sized clusters at Ottawa Linux Symposium 2007 
http://ltt.polymtl.ca/papers/bligh-Reprint.pdf

Now for some performance impact :

Here are some results I have taken comparing the optimized markers
approach with the dynamic ftrace approach. These runs with some ALU work
in tight loops, using clflush() to flush the cache lines pointing to
"global" data (pointer read : current->pid) used in the loop. I also
have the numbers for running the loop without the ALU work, but I leave
them out since they only make the tables harder to read : basically, the
cached impact for running the empty loop with markers or ftrace
instrumentation is about 0 to 3 cycles. It's the uncached impact which
clearly makes the difference between both approaches.

On AMD64, adding the markers or ftrace statement actually accelerates
the runs when executed with an ALU work baseline. It adds 1 to 2 cycles
with executed alone in the loop without any work.

Frank Ch. Eigler is preparing some macrobenchmarks. I hope he will find
time to post them soon.

Results in cycles per loop

baseline :
Cycles for ALU loop                    28.10013
(will be substracted for cached runs)

Cycles for clflush() and ALU loop     230.11087
(will be substracted from non-cached runs)

gcc version 4.1.3 20070812 (prerelease) (Debian 4.1.2-15), -O2

------------------------------------------------------------------------------
|x86 Pentium 4, 3.0GHz, Linux 2.6.25-rc7           |  cached    |  uncached  |
------------------------------------------------------------------------------
|Added cycles for optimized marker                 |  0.002     |   0.07     |
|Added cycles for normal marker                    |  0.004     | 154.7      |
|Added cycles for stack setup + (1+4 bytes) NOPs   |            |            |
|(6 local vars)                                    |  0.035     |   0.6      |
|Added cycles for stack setup + (1+4 bytes) NOPs   |            |            |
|(1 pointer read, 5 local vars)                    |  0.030     | 222.8      |
------------------------------------------------------------------------------

Results in cycles per loop

baseline :
Cycles for ALU and loop                 25.32369
(will be substracted for cached runs)

Cycles for clflush() and ALU loop      118.24227
(will be substracted from non-cached runs)

gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21), -O2

------------------------------------------------------------------------------
|AMD64, 2.0GHz, Linux 2.6.25-rc7                   |  cached    |  uncached  |
------------------------------------------------------------------------------
|Added cycles for optimized marker                 | -1.0       |    0.2     |
|Added cycles for normal marker                    | -0.3       |   41.8     |
|Added cycles for stack setup + (1+4 bytes) NOPs   | -0.5       |    0.01    |
|(6 local vars)                                    |            |            |
|Added cycles for stack setup + (1+4 bytes) NOPs   |  2.7       |   51.8     |
|(1 pointer read, 5 local vars)                    |            |            |
------------------------------------------------------------------------------

test bench at : http://ltt.polymtl.ca/svn/markers-test/

Regards,

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[RFC PATCH 1/2] Marker probes in futex.c, K. Prasad, (Tue Apr 15, 4:53 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Tue Apr 15, 5:02 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Tue Apr 15, 5:32 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Alexey Dobriyan, (Tue Apr 15, 5:50 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Tue Apr 15, 5:56 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Ingo Molnar, (Tue Apr 15, 6:17 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Tue Apr 15, 6:25 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Tue Apr 15, 6:47 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Ingo Molnar, (Tue Apr 15, 7:04 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Frank Ch. Eigler, (Tue Apr 15, 7:21 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Tue Apr 15, 7:39 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, K. Prasad, (Tue Apr 15, 8:52 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, K. Prasad, (Tue Apr 15, 9:13 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Ingo Molnar, (Tue Apr 15, 9:48 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Tue Apr 15, 2:38 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Ingo Molnar, (Wed Apr 16, 6:17 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Arjan van de Ven, (Wed Apr 16, 6:46 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Wed Apr 16, 7:00 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Arjan van de Ven, (Wed Apr 16, 7:24 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Ingo Molnar, (Wed Apr 16, 7:29 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Wed Apr 16, 7:48 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Arjan van de Ven, (Wed Apr 16, 8:32 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Wed Apr 16, 8:39 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Wed Apr 16, 8:51 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Wed Apr 16, 8:51 am)
text_poke, vmap and vmalloc on x86_64, Mathieu Desnoyers, (Wed Apr 16, 1:10 pm)
Re: text_poke, vmap and vmalloc on x86_64, Mathieu Desnoyers, (Wed Apr 16, 2:22 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Frank Ch. Eigler, (Thu Apr 17, 12:19 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Thu Apr 17, 1:16 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Frank Ch. Eigler, (Thu Apr 17, 3:02 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Thu Apr 17, 11:46 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Thu Apr 17, 11:56 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Andrew Morton, (Fri Apr 18, 3:44 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Frank Ch. Eigler, (Fri Apr 18, 7:29 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, K. Prasad, (Sat Apr 19, 5:13 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, K. Prasad, (Sat Apr 19, 5:28 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Sat Apr 19, 7:13 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Sat Apr 19, 7:52 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Sat Apr 19, 7:56 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Frank Ch. Eigler, (Sat Apr 19, 11:03 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Peter Zijlstra, (Sat Apr 19, 11:29 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Frank Ch. Eigler, (Sat Apr 19, 11:48 am)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Mathieu Desnoyers, (Sat Apr 19, 2:33 pm)
Re: [RFC PATCH 1/2] Marker probes in futex.c, Nicholas Miell, (Tue Apr 22, 10:50 am)