Documentation/trace/ftrace.txt says

  buffer_size_kb:

        This sets or displays the number of kilobytes each CPU
        buffer can hold. The tracer buffers are the same size
        for each CPU. The displayed number is the size of the
        CPU buffer and not total size of all buffers. The
        trace buffers are allocated in pages (blocks of memory
        that the kernel uses for allocation, usually 4 KB in size).
        If the last page allocated has room for more bytes
        than requested, the rest of the page will be used,
        making the actual allocation bigger than requested.
        ( Note, the size may not be a multiple of the page size
          due to buffer management overhead. )

        This can only be updated when the current_tracer
        is set to "nop".

But it's incorrect. currently total memory consumption is
'buffer_size_kb x CPUs x 2'.

Why two times difference is there? because ftrace implicitly allocate
the buffer for max latency too.

That makes sad result when admin want to use large buffer. (If admin
want full logging and makes detail analysis). example, If admin
have 24 CPUs machine and write 200MB to buffer_size_kb, the system
consume ~10GB memory (200MB x 24 x 2). umm.. 5GB memory waste is
usually unacceptable.

Fortunatelly, almost all users don't use max latency feature.
The max latency buffer can be disabled easily.

This patch shrink buffer size of the max latency buffer if
unnecessary.

Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
 kernel/trace/trace.c              |   22 +++++++++++++---------
 kernel/trace/trace.h              |    1 +
 kernel/trace/trace_irqsoff.c      |    9 +++++++++
 kernel/trace/trace_sched_wakeup.c |    9 +++++++++
 4 files changed, 32 insertions(+), 9 deletions(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 8683dec..ab2c061 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -2782,7 +2782,7 @@ int tracer_init(struct tracer ...
[ message continues ]

Hi Kosaki,

FYI, could you send emails to my goodmis account. I can easily miss
emails sent to my RH account since it is usually flooded with RH
Bugzilla reports.

(more below)


Actually, what would be better is to add a "use_max_tr" field to the
struct tracer in trace.h.  Then the latency tracers (irqsoff,
preemptoff, preemptirqsoff, wakeup, and wakeup_rt) can have this field
set.

Then, we can resize or even remove the max ring buffer when the
"use_max_tr" is not set (and on bootup). On enabling a latency tracer,
we can allocate the buffer. When we enable another tracer (or nop) if
the use_max_tr is not set, then we can remove the buffer.

Would you be able to do something like that?

Thanks,

-- Steve


--

[ message continues ]

Ahh sorry. yes, I had hear the same thing. I did forgot it.



How about this? 



From d17bef4652a00c124940cb8c28ff7fcf584c9008 Mon Sep 17 00:00:00 2001
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Date: Thu, 1 Jul 2010 13:06:57 +0900
Subject: [PATCH 1/2] tracing: shrink max latency ringbuffer if unnecessary

Documentation/trace/ftrace.txt says

  buffer_size_kb:

        This sets or displays the number of kilobytes each CPU
        buffer can hold. The tracer buffers are the same size
        for each CPU. The displayed number is the size of the
        CPU buffer and not total size of all buffers. The
        trace buffers are allocated in pages (blocks of memory
        that the kernel uses for allocation, usually 4 KB in size).
        If the last page allocated has room for more bytes
        than requested, the rest of the page will be used,
        making the actual allocation bigger than requested.
        ( Note, the size may not be a multiple of the page size
          due to buffer management overhead. )

        This can only be updated when the current_tracer
        is set to "nop".

But it's incorrect. currently total memory consumption is
'buffer_size_kb x CPUs x 2'.

Why two times difference is there? because ftrace implicitly allocate
the buffer for max latency too.

That makes sad result when admin want to use large buffer. (If admin
want full logging and makes detail analysis). example, If admin
have 24 CPUs machine and write 200MB to buffer_size_kb, the system
consume ~10GB memory (200MB x 24 x 2). umm.. 5GB memory waste is
usually unacceptable.

Fortunatelly, almost all users don't use max latency feature.
The max latency buffer can be disabled easily.

This patch shrink buffer size of the max latency buffer if
unnecessary.

Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
 kernel/trace/trace.c              |   39 ++++++++++++++++++++++++++++++------
 kernel/trace/trace.h              |    1 +
 ...
[ message continues ]

Does we need to shrink it when tracer_init() fails?
Although tracer_init() hardly fails, and there is no bad effect even we don't shrink it.
--

[ message continues ]

Nope. brief code of tracing_set_tracer() is here

========================================
        if (current_trace && current_trace->reset)
                current_trace->reset(tr);

        destroy_trace_option_files(topts);

        current_trace = t;

        topts = create_trace_option_files(current_trace);

        if (t->init) {
                ret = tracer_init(t, tr);
                if (ret)
                        goto out;
        }
========================================

That's mean, if t->init fail, we can't rollback old tracer. so your
suggested micro optimization
doesn't makes observable improvement, I think.

Thanks.
--

[ message continues ]

Commit-ID:  ef710e100c1068d3dd5774d2b34c5485219e06ce
Gitweb:     http://git.kernel.org/tip/ef710e100c1068d3dd5774d2b34c5485219e06ce
Author:     KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
AuthorDate: Thu, 1 Jul 2010 14:34:35 +0900
Committer:  Steven Rostedt <rostedt@goodmis.org>
CommitDate: Wed, 21 Jul 2010 10:20:17 -0400

tracing: Shrink max latency ringbuffer if unnecessary

Documentation/trace/ftrace.txt says

  buffer_size_kb:

        This sets or displays the number of kilobytes each CPU
        buffer can hold. The tracer buffers are the same size
        for each CPU. The displayed number is the size of the
        CPU buffer and not total size of all buffers. The
        trace buffers are allocated in pages (blocks of memory
        that the kernel uses for allocation, usually 4 KB in size).
        If the last page allocated has room for more bytes
        than requested, the rest of the page will be used,
        making the actual allocation bigger than requested.
        ( Note, the size may not be a multiple of the page size
          due to buffer management overhead. )

        This can only be updated when the current_tracer
        is set to "nop".

But it's incorrect. currently total memory consumption is
'buffer_size_kb x CPUs x 2'.

Why two times difference is there? because ftrace implicitly allocate
the buffer for max latency too.

That makes sad result when admin want to use large buffer. (If admin
want full logging and makes detail analysis). example, If admin
have 24 CPUs machine and write 200MB to buffer_size_kb, the system
consume ~10GB memory (200MB x 24 x 2). umm.. 5GB memory waste is
usually unacceptable.

Fortunatelly, almost all users don't use max latency feature.
The max latency buffer can be disabled easily.

This patch shrink buffer size of the max latency buffer if
unnecessary.

Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
LKML-Reference: <20100701104554.DA2D.A69D9226@jp.fujitsu.com>
Signed-off-by: ...
[ message continues ]