login
Search
Hi, This patch series provides the commands 'perf bts record' and 'perf bts trace'. These commands can record and analyze BTS (Branch Trace Store) log. And, provide the interface to use BTS log for application developers. BTS is a facility of Intel x86 processors, which can record the address of 'branch to/from' on every branch/jump instruction and interrupt. This facility is very useful for developers to test their software. For example, coverage test, execution path analysis, dynamic step count ...etc. Also, the tools have a very big advantage that they will require no changes to the target executable binaries. But, there are few applications using BTS. Reasons I guess are ... - Few people know what BTS is. - Few people know how to use BTS on Linux box. - It's hard to analyze the BTS log because it includes just a series of addresses. So, I want to provide a user-friendly interface to BTS for application developers. About new sub commands ======================== 'perf bts record' provides the easy way to record bts log. Usage is 'perf bts record <command>'. This command is just an alias to 'perf record -e branches:u -c 1 <command>'. But, new one is more simple and more intuitive. 'perf bts trace' can parse and analyze recorded bts log and print various information of execution path. This command can show address, pid, command name, function+offset, file path of elf. You can choose the printed information with option. Example: 'perf bts trace' function+offset irq_return+0x0 => _start+0x0 irq_return+0x0 => _start+0x0 _start+0x3 => _dl_start+0x0 irq_return+0x0 => _dl_start+0x0 irq_return+0x0 => _dl_start+0x26 irq_return+0x0 => _dl_start+0x2d _dl_start+0x71 => _dl_start+0x93 _dl_start+0x97 => _dl_start+0x78 ... This is the default behavior of 'perf bts trace'. It prints function+offset. Example2: 'perf bts -cas trace' command address function+offset ls 0xffffffff8146fe0e irq_return+0x0 => ls ...
Add new macro OPT_CALLBACK_DEFAULT_NOOPT for parse_options. It enables to pass the default value (opt->defval) to the callback function processing options require no argument. Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: linux-kernel@vger.kernel.org --- tools/perf/util/parse-options.h | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/tools/perf/util/parse-options.h b/tools/perf/util/parse-options.h index c7d72dc..abc31a1 100644 --- a/tools/perf/util/parse-options.h +++ b/tools/perf/util/parse-options.h @@ -119,6 +119,10 @@ struct option { { .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l), .value = (v), (a), .help = (h), .callback = (f), .flags = PARSE_OPT_NOARG } #define OPT_CALLBACK_DEFAULT(s, l, v, a, h, f, d) \ { .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l), .value = (v), (a), .help = (h), .callback = (f), .defval = (intptr_t)d, .flags = PARSE_OPT_LASTARG_DEFAULT } +#define OPT_CALLBACK_DEFAULT_NOOPT(s, l, v, a, h, f, d) \ + { .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l),\ + .value = (v), (a), .help = (h), .callback = (f), .defval = (intptr_t)d,\ + .flags = PARSE_OPT_LASTARG_DEFAULT | PARSE_OPT_NOARG} /* parse_options() will filter out the processed options and leave the * non-option argments in argv[]. --
This could be useful for perf probe too. :) -- Masami HIRAMATSU 2nd Dept. Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com --
Commit-ID: e4e18d568b0e833c75c1f7833e1690cdde8f4d76 Gitweb: http://git.kernel.org/tip/e4e18d568b0e833c75c1f7833e1690cdde8f4d76 Author: Akihiro Nagai <akihiro.nagai.hw@hitachi.com> AuthorDate: Fri, 3 Dec 2010 12:58:53 +0900 Committer: Arnaldo Carvalho de Melo <acme@redhat.com> CommitDate: Mon, 6 Dec 2010 15:33:29 -0200 perf options: add OPT_CALLBACK_DEFAULT_NOOPT Add new macro OPT_CALLBACK_DEFAULT_NOOPT for parse_options. It enables to pass the default value (opt->defval) to the callback function processing options require no argument. Reviewed-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> LKML-Reference: <20101203035853.7827.17502.stgit@localhost6.localdomain6> Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> --- tools/perf/util/parse-options.h | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/tools/perf/util/parse-options.h b/tools/perf/util/parse-options.h index c7d72dc..abc31a1 100644 --- a/tools/perf/util/parse-options.h +++ b/tools/perf/util/parse-options.h @@ -119,6 +119,10 @@ struct option { { .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l), .value = (v), (a), .help = (h), .callback = (f), .flags = PARSE_OPT_NOARG } #define OPT_CALLBACK_DEFAULT(s, l, v, a, h, f, d) \ { .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l), .value = (v), (a), .help = (h), .callback = (f), .defval = (intptr_t)d, .flags = PARSE_OPT_LASTARG_DEFAULT } +#define OPT_CALLBACK_DEFAULT_NOOPT(s, l, v, a, h, f, d) \ + { .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l),\ + .value = (v), (a), .help = (h), .callback = (f), .defval = (intptr_t)d,\ + .flags = PARSE_OPT_LASTARG_DEFAULT | PARSE_OPT_NOARG} /* parse_options() will filter out the processed options and leave the * ...
Introduce the easy way to record bts log, 'perf bts record'.
This command can record the bts log while specified command is executing,
and save to the file "perf.data"
Usage:
perf bts record <tracee command>
Example:
# perf bts record ls -l
(ls -l outputs)
# perf record: Captured and wrote 4.381 MB perf.data (~191405 samples)
# ls
perf.data
Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
Cc: linux-kernel@vger.kernel.org
---
tools/perf/Documentation/perf-bts.txt | 24 +++++++++++++
tools/perf/Makefile | 1 +
tools/perf/builtin-bts.c | 62 +++++++++++++++++++++++++++++++++
tools/perf/builtin.h | 1 +
tools/perf/perf.c | 1 +
5 files changed, 89 insertions(+), 0 deletions(-)
create mode 100644 tools/perf/Documentation/perf-bts.txt
create mode 100644 tools/perf/builtin-bts.c
diff --git a/tools/perf/Documentation/perf-bts.txt b/tools/perf/Documentation/perf-bts.txt
new file mode 100644
index 0000000..55a2fe6
--- /dev/null
+++ b/tools/perf/Documentation/perf-bts.txt
@@ -0,0 +1,24 @@
+perf-bts(1)
+==============
+
+NAME
+----
+perf-bts - Record branch-trace-store log
+
+SYNOPSIS
+--------
+[verse]
+'perf bts' record <command>
+
+DESCRIPTION
+-----------
+This command can record branch-trace-store log.
+Branch-trace-store is a facility of processors. It can record
+address of branch to/from on every branch instruction and interrupt.
+
+'perf bts record <command>' records branch-trace-store log while specified
+command is executing. And, save to the file "perf.data".
+
+SEE ALSO
+--------
+linkperf:perf-record[1]
diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index b3e6bc6..14de491 100644
--- ...Good work :) Just update tools/perf/command-list.txt, so that perf shows -- Masami HIRAMATSU 2nd Dept. Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com --
Introduce new sub command 'perf bts trace'. This command can parse and print bts log recorded by 'perf bts record'. Usage: - First, record the bts log 'perf bts record <command>' - Second, parse and print bts log 'perf bts trace' Output: 0xffffffff8146fe0e => 0x0000003806200b20 0x0000003806200b23 => 0x0000003806204910 0xffffffff8146fe0e => 0x0000003806204910 0xffffffff8146fe0e => 0x0000003806204936 0xffffffff8146fe0e => 0x000000380620493d 0x0000003806204981 => 0x00000038062049a3 0x00000038062049a7 => 0x0000003806204988 ... Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: linux-kernel@vger.kernel.org --- tools/perf/Documentation/perf-bts.txt | 14 ++++++-- tools/perf/builtin-bts.c | 59 +++++++++++++++++++++++++++++++-- 2 files changed, 67 insertions(+), 6 deletions(-) diff --git a/tools/perf/Documentation/perf-bts.txt b/tools/perf/Documentation/perf-bts.txt index 55a2fe6..5920dcc 100644 --- a/tools/perf/Documentation/perf-bts.txt +++ b/tools/perf/Documentation/perf-bts.txt @@ -3,22 +3,30 @@ perf-bts(1) NAME ---- -perf-bts - Record branch-trace-store log +perf-bts - Record and print branch-trace-store log SYNOPSIS -------- [verse] -'perf bts' record <command> +'perf bts' [<options>] {record|trace} DESCRIPTION ----------- -This command can record branch-trace-store log. +This command can record and print branch-trace-store log. Branch-trace-store is a facility of processors. It can record address of branch to/from on every branch instruction and interrupt. 'perf bts record <command>' records branch-trace-store log while specified command is executing. And, save to the file "perf.data". +'perf bts trace' parses recorded branch-trace-store log and prints ...
It looks a good step. -- Masami HIRAMATSU 2nd Dept. Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com --
Provide the function of printing pid and command name to
'perf bts trace'. Users can select items to print with options.
For example,
'perf bts -p trace' prints only pid,
'perf bts -ac trace' prints address and comamnd name.
'perf bts trace' prints only address (default)
This is output sample (perf bts -ac trace):
command address
ls 0xffffffff8146fe0e => ls 0x0000003806200b20
ls 0xffffffff8146fe0e => ls 0x0000003806200b20
ls 0x0000003806200b23 => ls 0x0000003806204910
ls 0xffffffff8146fe0e => ls 0x0000003806204910
ls 0xffffffff8146fe0e => ls 0x0000003806204936
ls 0xffffffff8146fe0e => ls 0x000000380620493d
ls 0x0000003806204981 => ls 0x00000038062049a3
ls 0x00000038062049a7 => ls 0x0000003806204988
Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
Cc: linux-kernel@vger.kernel.org
---
tools/perf/Documentation/perf-bts.txt | 18 ++++-
tools/perf/builtin-bts.c | 124 ++++++++++++++++++++++++++++++++-
2 files changed, 135 insertions(+), 7 deletions(-)
diff --git a/tools/perf/Documentation/perf-bts.txt b/tools/perf/Documentation/perf-bts.txt
index 5920dcc..56ddaa4 100644
--- a/tools/perf/Documentation/perf-bts.txt
+++ b/tools/perf/Documentation/perf-bts.txt
@@ -3,7 +3,7 @@ perf-bts(1)
NAME
----
-perf-bts - Record and print branch-trace-store log
+perf-bts - Record and analyze branch-trace-store log
SYNOPSIS
--------
@@ -12,20 +12,32 @@ SYNOPSIS
DESCRIPTION
-----------
-This command can record and print branch-trace-store log.
+This command can record and analyze branch-trace-store log.
Branch-trace-store is a facility of ...Looks good to me :) -- Masami HIRAMATSU 2nd Dept. Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com --
Provide the function to print file path to the executed elf.
Users can enable it with option '-e' or '--elfpath'.
For example,
'perf bts -ae trace'
This command prints address and file path of elf.
And, output is:
address elf_filepath
0xffffffff8146fe0e /lib/modules/2.6.37-rc2-tip+/build/vmlinux => 0x00007fd4038e3b20 /lib64/ld-2.12.90.so
0xffffffff8146fe0e /lib/modules/2.6.37-rc2-tip+/build/vmlinux => 0x00007fd4038e3b20 /lib64/ld-2.12.90.so
0x00007fd4038e3b23 /lib64/ld-2.12.90.so => 0x00007fd4038e7910 /lib64/ld-2.12.90.so
0xffffffff8146fe0e /lib/modules/2.6.37-rc2-tip+/build/vmlinux => 0x00007fd4038e7910 /lib64/ld-2.12.90.so
0xffffffff8146fe0e /lib/modules/2.6.37-rc2-tip+/build/vmlinux => 0x00007fd4038e7936 /lib64/ld-2.12.90.so
0xffffffff8146fe0e /lib/modules/2.6.37-rc2-tip+/build/vmlinux => 0x00007fd4038e793d /lib64/ld-2.12.90.so
0x00007fd4038e7981 /lib64/ld-2.12.90.so => 0x00007fd4038e79a3 /lib64/ld-2.12.90.so
0x00007fd4038e79a7 /lib64/ld-2.12.90.so => 0x00007fd4038e7988 /lib64/ld-2.12.90.so
...
Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
Cc: linux-kernel@vger.kernel.org
---
tools/perf/Documentation/perf-bts.txt | 3 +++
tools/perf/builtin-bts.c | 31 +++++++++++++++++++++++++++++++
2 files changed, 34 insertions(+), 0 deletions(-)
diff --git a/tools/perf/Documentation/perf-bts.txt b/tools/perf/Documentation/perf-bts.txt
index 56ddaa4..acabffc 100644
--- a/tools/perf/Documentation/perf-bts.txt
+++ b/tools/perf/Documentation/perf-bts.txt
@@ -38,6 +38,9 @@ OPTIONS
-p::
--pid::
Print pid.
+-e::
+--elfpath::
+ Print file path of executed elf.
SEE ALSO
--------
diff --git a/tools/perf/builtin-bts.c b/tools/perf/builtin-bts.c
index 11e491b..3b2a21e ...Looks good to me :) -- Masami HIRAMATSU 2nd Dept. Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com --
For ease of use, add option printing all information '-A' or '--all'. This option can print following information. - pid - command name - address - function+offset - elf file path Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: linux-kernel@vger.kernel.org --- tools/perf/Documentation/perf-bts.txt | 3 +++ tools/perf/builtin-bts.c | 6 ++++++ 2 files changed, 9 insertions(+), 0 deletions(-) diff --git a/tools/perf/Documentation/perf-bts.txt b/tools/perf/Documentation/perf-bts.txt index 13ee862..c598dd2 100644 --- a/tools/perf/Documentation/perf-bts.txt +++ b/tools/perf/Documentation/perf-bts.txt @@ -44,6 +44,9 @@ OPTIONS -s:: --symbol:: Print function name and offset. (default) +-A:: +--all:: + Print all information. SEE ALSO -------- diff --git a/tools/perf/builtin-bts.c b/tools/perf/builtin-bts.c index 71c7fbe..c359f70 100644 --- a/tools/perf/builtin-bts.c +++ b/tools/perf/builtin-bts.c @@ -34,6 +34,9 @@ struct exec_info { #define EI_FLAG_PRINT_ELFPATH (1 << 3) #define EI_FLAG_PRINT_SYMBOL (1 << 4) +/* all print flags are enabled */ +#define EI_FLAG_PRINT_ALL -1UL + /* it's used when no print item specified */ #define EI_FLAG_PRINT_DEFAULT EI_FLAG_PRINT_SYMBOL @@ -92,6 +95,9 @@ static const struct option bts_options[] = { "print function+offset (default)", set_print_flags, (void *)EI_FLAG_PRINT_SYMBOL), + OPT_CALLBACK_DEFAULT_NOOPT('A', "all", NULL, NULL, + "print all items", set_print_flags, + (void *)EI_FLAG_PRINT_ALL), OPT_END() }; --
Looks good to me :) -- Masami HIRAMATSU 2nd Dept. Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com --
Provide the function to print function+offset. And, set it as the default behavior of 'perf bts trace'. To use this function, users can also specify the option '-s' or '--symbol'. Example: 'perf bts -as trace' This command prints address and function+offset. Output sample: address function+offset 0xffffffff8146fe0e irq_return+0x0 => 0x00007fd4038e3b20 _start+0x0 ... 0x000000380661ee79 __libc_start_main+0xf9 => 0x00000000004035c3 main+0x0 0xffffffff8146ef4e irq_return+0x0 => 0x00000000004035c3 main+0x0 0x00000000004035e8 main+0x25 => 0x000000000040bca0 set_program_name+0x0 0xffffffff8146ef4e irq_return+0x0 => 0x000000000040bca0 set_program_name+0x0 0x000000000040bcae set_program_name+0xe => 0x00000000004023d0 strrchr@plt+0x0 0x00000000004023d0 strrchr@plt+0x0 => 0x00000000004023d6 strrchr@plt+0x6 ... 0x0000000000403e0c main+0x849 => 0x00000000004021f0 exit@plt+0x0 0x00000000004021f0 exit@plt+0x0 => 0x00000000004021f6 exit@plt+0x6 0x00000000004021fb exit@plt+0xb => 0x00000000004020d0 _init+0x18 0x00000000004020d6 _init+0x1e => 0x00000038062149d0 _dl_runtime_resolve+0x0 ... Signed-off-by: Akihiro Nagai <akihiro.nagai.hw@hitachi.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: linux-kernel@vger.kernel.org --- tools/perf/Documentation/perf-bts.txt | 5 +++- tools/perf/builtin-bts.c | 44 ++++++++++++++++++++++++++++++--- 2 files changed, 44 insertions(+), 5 deletions(-) diff --git a/tools/perf/Documentation/perf-bts.txt b/tools/perf/Documentation/perf-bts.txt index acabffc..13ee862 100644 --- a/tools/perf/Documentation/perf-bts.txt +++ b/tools/perf/Documentation/perf-bts.txt @@ -31,7 +31,7 @@ OPTIONS Specify input file name to analyze. -a:: --addr:: - Print ...
What would be very nice output is something like a source-code browser (tig might have something worth borrowing) and annotating the output using the bts output. You could for example darken the code that didn't get any coverage, and print a percentage for each conditional statement. That way you can easily see how your code is traversed. --
We have developed the coverage test tool and execution path analyzer using BTS called "Btrax". It can show the executed/unexecuted source codes with color. Please refer this slide about details of Btrax. http://lca2009.linux.org.au/slides/52.pdf The purpose of 'perf bts' is to implement similar functions to Btrax. And also, I'll try to reduce memory copies from kernel space to user, because perf copies 3 times. But btrax does just one copy. Please see the following benchmark results. Test program: compile with -O0 option this source code int main(void) { int i; while(i++ < 100000000); return 0; } Execution Results: Native: # time ./a.out real 0m0.349s user 0m0.346s sys 0m0.002s Btrax: (enabling BTS_OFF_OS flag for this benchmark) (kernel 2.6.30 on RHEL 5.4) # bt_collect_log -d log -c ./a.out real 1m19.950s user 0m14.783s sys 0m40.655s (trace data size: 1,537,307,064 bytes) perf: # time perf bts record ./a.out Processed 0 events and LOST 5002! Check IO/CPU overload! # perf record: Captured and wrote 2704.231 MB perf.data (~118149662 samples) real 2m1.885s user 0m7.184s sys 0m19.194s Bench Mark Enviroment: Native and perf: latest -tip kernel on Fedora14 Btrax: vanilla 2.6.30 on RHEL5.4 Core2Duo E6550 @2.33GHz Mem DDR2 4GB --
There is another kind of mode that I suspect would be very useful: something like
a userspace function graph tracer (you can have a look into the kernel function
graph tracer we have in ftrace to get an overview).
So, the idea would be to rebuild the whole function call flow:
main() {
func1()
func2() {
func3()
}
}
This would require to deref the instructions into the dso addresses from
the bts trace, keep only the "call" and the "ret" (there would be a small arch
backend for this mode) and rebuild the whole tree of calls.
We already have all the dso address mapping API in place with perf.
I really think this could be a very useful tool. Also we can even later
expand this to the branches, as Peter suggested. But starting with calls
would a very great start already.
Thanks.
--
You might think the next step is displaying line number of the address. Feel free to use/update probe-finder.c :) -- Masami HIRAMATSU 2nd Dept. Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com --
| Jesse Barnes | Re: [stable] [BUG][PATCH] cpqphp: fix kernel NULL pointer dereference |
| Greg KH | [003/136] p54usb: add Zcomax XG-705A usbid |
| Magnus Damm | [PATCH 03/07] ARM: Use shared GIC entry macros on Realview |
| Oliver Neukum | Re: [Bug #13682] The webcam stopped working when upgrading from 2.6.29 to 2.6.30 |
| Martin Schwidefsky | Re: [PATCH] optimized ktime_get[_ts] for GENERIC_TIME=y |
git: | |
| Junio C Hamano | Re: Some advanced index playing |
| Jeff King | Re: confusion over the new branch and merge config |
| Robin Rosenberg | Re: cvs2svn conversion directly to git ready for experimentation |
| Linus Torvalds | git binary size... |
| Ævar Arnfjörð Bjarmason | Re: Challenge with Git-Bash |
| Linux Kernel Mailing List | md: move allocation of ->queue from mddev_find to md_probe |
| Linux Kernel Mailing List |