Re: [RFC PATCH 1/3] Unified trace buffer

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Mathieu Desnoyers
Date: Thursday, September 25, 2008 - 10:15 am

* Linus Torvalds (torvalds@linux-foundation.org) wrote:

We could use a page header instead to contain the "unused_size"
information. It does not need to be an event per se. Putting this
information in the page header makes it easy for a consumer to just read
the amount of bytes needed, excluding the padding (turns up to be useful
for network streaming of trace data). Also, it frees up one event ID for
other uses. I think the event ID real estate is pretty important,
because every event ID we don't keep for "internal uses" could be used
to encode standard tracer event IDs.



I would prefer to put the extended timestamp within the event header
instead of creating a separate entry for this for atomicity concerns
(what happens if a long interrupt executes between the TSCExtend marker
event and the event expecting to be written right next to it ?). If we
choose to have such "full tsc" event headers, we would have to reserve 1
selection bit. It leaves us 4 bits for event IDs. If we remove
heartbeats, we need even less "internal" IDs.

Given that we can allocate event IDs per buffer, if in general we assume
that buffer users will have few event IDs, I think we can find a way to
minimize the number of "internal" event IDs and keep possibly all the 4
bits (16 IDs) for real tracer events. One way to achieve it is to encode
the extra typing information within a table (dumped in a separate buffer)
indexed by event ID.



This one could even be a standard event put in a single buffer. There is
no need to repeat it in various buffers all over the place.


I agree that exporting semantic information is important. Moreover, I
think this should also be made available when the trace is exported in
binary format to userspace. The markers currently in mainline has been
designed to do this efficiently. With small adaptation of the markers,
one could do a :

  trace_mark(block, read,
             "Reading sector sec_from to sec_to",
             "sec_from %lu sec_to %lu",
             sector, sector + nsec);

The nice part about the markers is that it keeps tables for the
description "Reading sector sec_from to sec_to" and the typing
information "sec_from %lu sec_to %lu" in a separate section. Therefore,
we only need to dump this information once (at trace start or when the
module containing this specific marker is loaded). It automatically
deals with module load/unload and does not require to write their format
string in the trace buffers.

The strings can be looked up by a userspace pretty-printer by dumping a
table mapping event IDs to marker names, which in turn map to
description and event types. This implies creating a small "metadata"
buffer along with each data transfer buffer to export these tables. This
metadata buffer must be read first to get the event typing information,
and then the userspace program is all set to pretty-print the binary
information.

An efficient ID -> format string mapping could also be kept around in
the kernel (or built on demand) to simplify the task for an in-kernel
pretty-printer.

Mathieu


-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Tue Sep 23, 10:10 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Peter Zijlstra, (Wed Sep 24, 8:03 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 8:44 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 8:47 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Peter Zijlstra, (Wed Sep 24, 9:11 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Wed Sep 24, 9:13 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 9:24 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 9:26 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 9:31 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 9:37 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Peter Zijlstra, (Wed Sep 24, 9:39 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 9:49 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Wed Sep 24, 9:51 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 9:56 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 10:25 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 10:36 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 10:49 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 10:54 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Wed Sep 24, 11:01 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 11:04 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 1:23 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 1:30 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, David Miller, (Wed Sep 24, 1:37 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 1:39 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 1:47 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 1:48 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 1:49 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 1:51 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Wed Sep 24, 1:53 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 1:56 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Wed Sep 24, 2:03 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 2:08 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 2:17 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Frank Ch. Eigler, (Wed Sep 24, 2:24 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 2:33 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Wed Sep 24, 2:51 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 3:28 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Wed Sep 24, 3:41 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 3:38 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Peter Zijlstra, (Thu Sep 25, 3:41 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Thu Sep 25, 7:33 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Peter Zijlstra, (Thu Sep 25, 7:53 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 8:05 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 8:20 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Thu Sep 25, 8:25 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 8:26 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 8:36 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Thu Sep 25, 9:23 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 9:32 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Thu Sep 25, 9:37 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 9:40 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 9:49 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 9:53 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 10:02 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 10:07 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Thu Sep 25, 10:15 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Thu Sep 25, 10:20 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 10:22 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 10:29 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 10:32 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 10:39 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Thu Sep 25, 10:42 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 11:14 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 12:55 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 1:12 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 1:20 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 1:24 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 1:29 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Mathieu Desnoyers, (Thu Sep 25, 1:29 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 1:47 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 1:52 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 2:01 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 2:02 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 2:10 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 2:14 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Martin Bligh, (Thu Sep 25, 2:15 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 2:16 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 2:41 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 2:55 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 2:56 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 2:58 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 3:14 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 3:25 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 3:39 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 3:45 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 3:55 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 3:59 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 4:04 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Thu Sep 25, 4:25 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Linus Torvalds, (Thu Sep 25, 4:33 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 6:17 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 6:27 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 6:27 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 6:35 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 6:49 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 7:07 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Thu Sep 25, 7:25 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Jeremy Fitzhardinge, (Thu Sep 25, 10:31 pm)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Fri Sep 26, 3:41 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Thomas Gleixner, (Fri Sep 26, 7:04 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Sat Sep 27, 10:16 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Sat Sep 27, 10:36 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Sat Sep 27, 10:38 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Peter Zijlstra, (Sat Sep 27, 10:50 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Steven Rostedt, (Sat Sep 27, 11:18 am)
Re: [RFC PATCH 1/3] Unified trace buffer, Ingo Molnar, (Sat Sep 27, 11:42 am)