Recently, the CE Linux forum has been working to revive the

Linux-tiny project.  At OLS, I asked for interested parties

to volunteer to become the new maintainer for the Linux-tiny patchset.
A few candidates came forward, but eventually Michael Opdenacker

was selected as the new primary maintainer.  A few other

people, including John Cooper of Wind River and myself

are working to support this effort.
Recently, many of the Linux-tiny patches have been brought up-to-date

and are now available for use with a 2.6.22 kernel.  The intent

is to test these, and begin mainlining the most effective sub-patches,

in the next few months.
Some automated testing has already been set up, with some

preliminary results published at a CELF conference in Japan.

(See the linux-tiny page below for a link to the presentation.)

Hopefully, results publishing will also be automated soon.
We encourage anyone with interest in this project to get involved.

If you have ideas how to reduce the static or dynamic memory footprint

of Linux, or, even better, patches for this, please let us know about

them.
Please see http://elinux.org/Linux_Tiny
A related document: http://elinux.org/Kernel_Size_Tuning_Guide

is undergoing an update this week.
Thanks,

 -- Tim
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-

Cool!
Could you update http://www.selenic.com/linux-tiny/ to mention the new 
I've been playing with an idea for a while to improve the printk() situation,

but it's a more intrusive change than I've had time to bang on.
Right now, the first argument to printk() is a loglevel, but it's handled via

string concatenation.  I'd like to change that to be an integer, and make it

an actual comma-separated first argument.  (Mandatory, not optional.)
So instead of:

  printk(KERN_NOTICE "Fruit=%d\n", banana);

It would now be:

  printk(KERN_NOTICE, "Fruit=%d\n", banana);
Change the header from:

  #define KERN_NOTICE "<5>"

to:

  #define KERN_NOTICE 5
Then you can change the printk guts to do something vaguely like (untested):

#define printk(arg1, arg2, ...) actual_printk("<" #arg1 ">" arg2, __VA_ARGS__)
And so far no behavior has changed.  But now the _fun_ part is, you can add a

config symbol for "what is the minimum loglevel I care about?"  Set that as a

number from 0-9.  And then you can define the printk to do:
#define printk(level, str, ...) \

  do { \

    if (level < CONFIG_PRINTK_DOICARE) \

      actual_printk("<" #level ">" str, __VA_ARGS__); \

  } while(0);
And viola (however you spell that, I think I'm using the stringed instrument

but it's french and I'm not trying to type a diacritical mark anyway), the

compiler's dead code eliminator zaps the printks you don't care about so they

don't bloat the kernel image.  But this doesn't _completely_ eliminate

printks, so you can still get the panic() calls and such.  You tweak precisly

how much bloat you want, using the granularity information that's already

there in the source code...
Opinions?
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

Assuming that we want to go down that road, I think you can do better with

more evil macro magic, by using something along the lines of
#define KERN_NOTICE "<5>",
#define PRINTK_CONTINUED "",
 #define printk(level, str, ...) \

   do { \

     if (sizeof(level) == 1) /* continued printk */\

	actual_printk(str, __VA_ARGS__); \

     else if ((level[1] - '0') < CONFIG_PRINTK_DOICARE) \

       actual_printk(level str, __VA_ARGS__); \

   } while(0);
Then you don't have to change every single printk in the kernel, but

only those that don't currently come with a log level. More importantly,

you can do the conversion without a flag day, by spreading (an empty)

PRINTK_CONTINUED in places that do need a printk without a log level.
	Arnd <><

-

The "change every printk in the kernel" suggestion came from me trying to

figure out how to get the printk() calls below a certain log level to

optimize out and not take up space in the binary.
The above doesn't address the original cause of the thread, as far as I can 
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

The problem is, how do you know whether to print a continued printk or not?

It depends on the loglevel of the first printk.
So besides compile-time parsing of the source code, replacing printk with

loglevel specific alternatives (one way or the other) seems the only option.
Greetings,
Indan
-

Those need to be looked at individually. You can normally see easily from

the context whether the missing log level was an accident, or the author

actually has multiple printk statements for a single line. In one case,

you would add a log level, in the other case, you can add PRINTK_CONTINUED,

or something similar. An alternative to PRINTK_CONTINUED might be a new

function, e.g. printk_continued() or similar that does not expect a log
That would mean replacing all of them, not just those that currently lack

a loglevel.
	Arnd <><

-

If think you misunderstood:

Say, you compile out everything of DEBUG level.

Say, you have a continued printk() after each and every pr_debug().
Then how is the macro supposed to know (at compile-time) that the

continued printk()  actually belongs to a DEBUG printk (and not another

one)?
	Bernd

--

Firmix Software GmbH                   http://www.firmix.at/

mobil: +43 664 4416156                 fax: +43 1 7890849-55

          Embedded Linux Development and Services
-

#if CONFIG_PRINTK_DOICARE >= 3

#define PRINTK_DEBUG_CONTINUED "",

#else

#define PRINTK_DEBUG_CONTINUED PRINTK_DEBUG

#endif

...
#define printk(level, str, ...) \

   do { \

     if (sizeof(level) == 1) /* continued printk */\

        actual_printk(str, __VA_ARGS__); \

     else if ((level[1] - '0') < CONFIG_PRINTK_DOICARE) \

       actual_printk(level str, __VA_ARGS__); \

   } while(0);
Now you can mark a continued KERN_ERR call as such.  And depending on

CONFIG_PRINTK_DOICARE it will either be printed as always or turned into

a stand-alone printk() that gets optimized away.  Do the same for all

the other levels.
Open question remains: do we want to go down that road?  I have no idea.
Jörn
--

I've never met a human being who would want to read 17,000 pages of

documentation, and if there was, I'd kill him to get him out of the

gene pool.

-- Joseph Costello

-

All the people who don't have the need will come up with reasons not to

do it. Like "saves too little" and "makes debugging hard" (these are the

people who don't realize that having no output device and human in the

loop makes it hard, too). How about "too complex, would confuse users,"

that's popular.
I could go into a list of thing people want to take out or keep out for

reasons which boil down to "I don't need it" or "leaving it out only

bothers lazy users who don't want to convert to {flavor_of_the_month}."
People with really small systems care about every few bytes, people with

big systems don't understand (in general) about people with small

systems. The best developers do, fortunately, and will probably approve

of kernel liposuction.
--

Bill Davidsen <davidsen@tmr.com>

   "We have more to fear from the bungling of the incompetent than from

the machinations of the wicked."  - from Slashdot

-

I agree in principal with the idea, but there are some major

practical wrinkles that would have to be worked through.
First, not all printks that are missing a log level should have one.

People do stuff like this:
printk(KERN_INFO "interesting info follows:");

...

printk("var5: %d\n", var5);
Or even things that evaluate to:

printk("");
The code inside printk currently has to examine the

strings, looking for line feeds and inserting log levels.
Given that there are about 60,000 printks in the kernel (and that's

not counting wrappers like dprintk() and other locally-defined

functions and macros) it would be a huge task to examine the code

and differentiate strings that really start a new log message

(and thus should have an attached log level) and strings

that don't.

 -- Tim
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-

I've converted most all of that treewide.
printk(KERN_<level> to pr_<level>(
It's pretty automated.
$ cat pr_alert.sh

#!/bin/sh

egrep -r -w --include=*.[ch] -l "printk[[:space:]]*\([[:space:]]*KERN_ALERT" * | \

 xargs perl ../cvt_pr.pl KERN_ALERT pr_alert
$ cat cvt_pr.pl

if ($#ARGV < 3) {

    print "usage: KERN_<level> pr_<level> files...\n";

    exit;

}
for ($i=2; $i<$#ARGV; $i++) {

    PrintkSearchReplace($ARGV[$i], $ARGV[0], $ARGV[1]);

}
sub PrintkSearchReplace{

    my($file, $search, $replace) = @_;
    my $content = "";

    local( $/ );

    open( my $fh, $file ) or die "File not found '$file'\n";

    $content = <$fh>;

    close(my $fh);

    my $orig = $content;
    $content =~ s/\bprintk[[:space:]]*\([[:space:]]*${search}[[:space:]]*([^\"]*)\"([^\\]*)\\n\"/${replace}\(\1 \"\2\"/mgs;

    $content =~ s/\b${replace}\( /${replace}\(/mgs;
    if ($orig ne $content)

    {

        open(my $fh, ">${file}") or die "Could not open '$file'\n";

        print $fh $content;

        close(my $fh);

    }

}
-

Perl, being a write-only language, does not help my poor little brain

understand what's going on.  You convert printk(KERN_INFO, blah) to

pr_INFO(blah)?  I'm not finding pr_INFO with a grep on the files in

2.6.23-rc7.  Is this something you added?
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

more or less.

printk(KERN_INFO foo) to pr_info(foo)

printk(KERN_EMERG foo) to pr_emerge(foo)
I haven't submitted them.
There's a lot of sensible resistance to what appears

to be churn.  Some of the resistance is historical,

merging large changes used to be much more painful

pre git, some is just simple resistance to change.
I started with submitting an add of pr_err to kernel.h

which Andrew Morton picked up for awhile, but dropped.
I've got a local tree with those changes.
for example:
KERN_EMERG  -> pr_emerg  is  ~100KB

KERN_ALERT  -> pr_alert  is   ~80KB

KERN_CRIT   -> pr_crit   is  ~200KB

KERN_NOTICE -> pr_notice is  ~400KB

KERN_INFO   -> pr_info   is ~2500KB
I intended to strip the "\n" trailer from the messages.
Back to the scripts:
In this case, there are multiple files.
A script that finds all the files that contain a search string,

and a perl script that effectively s/search/replace/g on those files.
sed didn't work as well as perl here because I wanted to

play with perl a bit and many printk(KERN_<level> foo)

function calls are split across multiple lines.
I've still got to show a real use to this change set.
I believe controlling the interleaving of log messages

by having multiple statement printks have a start and end,

choosing specific message levels in compiled code,

and choosing to print file/function/line per compiled

code block might do, but more utility to the changes

is probably necessary before it could be applied.
cheers, Joe
-

Hmmm.  The hard part isn't making printk(0,blah) mean the same as not having a

log level message now, because the current logic already handles it.  The

problem is that filtering continuations of previous messages involves knowing

what log level the previous message was so you know whether or not to filter

it.
Yeah, that would take some doing to untangle.  An incremental switchever (easy

printks first, I.E. the ones that currently specify a loglevel) seems more

strongly indicated...
That said, I started this by noting I haven't personally had time to bang on 
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

You have to jump through less hoops if you do:
#define KERN_NOTICE 5,
But the problem remains that there are printk's which don't have

a KERN_* as the first argument. Those are also impossible to get

rid off in this way, as the loglevel is unknown (and you don't want

partially printed messages).
So adding the comma is really needed and in addition all printk's

without a loglevel should get one. Which clutters the code and may

increase codesize.
A quick scroll through a vmlinux binary shows that there are quite a

lot areas consisting only of some repeated pattern. Mostly 0x00, but

also 0x90 and ".GCC: (GNU) 4.2.1.". Getting rid of those would save

something between 50 and 100KB.
Greetings,
Indan
-

Less change to the source, but the result is less obvious about what it's

doing.  I'd personally rather have the churn than wind up with magic 
It's ok to _explicitly_ not have a loglevel, and thus take a known default.

The problem is printing out less than a full line, continuing it later, and 
Worse, if you feed an absolute path to O= when you build the kernel out of

tree, then it uses absolute paths for all the __FILE__ strings and that makes

kernel BIIIIIG.  (Did that by accident a while back.)  Too bad there's no way

to keep the __FILE__ strings compressed at runtime and gunzip them as needed

like busybox does with help messages... :)
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

That's actually a fairly straightforward problem to solve.  Really we

ought to be able to guarantee that lines from different CPUs are not

intermixed.  This can end up in a log like this:
info: Current device state: <1>netdev watchdog timeout: eth0

reg1=0xABCD reg2=0xDEADBEEF
Clearly unpleasant, no?  Really any logging which wants to print out

a bunch of stuff with multiple printk()s should use an API something

like this:
Option 1: Buffer allocated with kmalloc().  Sleep-ish context, can  
Option 3: Preallocated per-cpu interrupts-only buffer.  Only from  
For all of the above, the final "qprintk_finish()" call would

essentially take the printk spinlock and write the contents of the

qpk->buffer into the dmesg ringbuffer, inserting ("<%d>", level) at

the beginning and after each newline.  The buffers would all be some

useful fixed size (a page?); if you need more than that then you are

probably trying to queue too much and excess data would be truncated.
Since the level would be a constant passed to qprintk_

{kmalloc,percpu,irq} in almost every case, you could easily do

something like this:
static inline int qprintk_kmalloc(struct printk_queue *qpk,

		unsigned int level, gfp_t gfp)

{

	if (level > CONFIG_MAX_LOG_LEVEL) {

		qpk->type = QPRINTK_TYPE_NONE; /* also 0 */

		qpk->buffer = NULL;

		return 0;

	}
	_qprintk_kmalloc(qpk, level, gfp);

}
#define qprintk(QPK, FMT...) do { \

		if ((QPK)->type)

			_qprintk(QPK, FMT);

	} while(0)
With a bit more glue that would cause GCC to notice that for a given

qprintk_kmalloc the "qpk->type" is always zero because the level is

too high, and therefore it would optimize out *ALL* of the

_qprintk_kmalloc(), _qprintk(), and _qprintk_finish() calls.
Cheers,

Kyle Moffett
-

A negative is that lockup conditions swallow partial messages.
Another approach that doesn't require any new buffering is:
id = printk_block_start();

printk_block(id, fmt, ...)

printk_block_end(id)
and have print_block output the id when multiple IDs are

concurrently issued.
This requires a trivial tool to post-process the log

when messages are interleaved.
-

But typically you don't care if a "partial line" gets swallowed

regardless.  The only reason people really use partial lines is when

they're accumulating a variable number of things into a single line

and so a single printk() won't do, and in that case it's really not a

problem to "lose" the first half of the line in event of a crash.

And hell, if it matters that much you could just make the qprintk_

{kmalloc,percpu,irq} functions chain the qpk variables on a little

linked list and stuff an smp_wmb() in the _gprint() function after

writing the text and before writing the size.  That way any panic

could very carefully look at the messages being queued during the

crash and attempt to write out partial buffers.
It's a technique which in combination with looking at the first 3

characters of the arguments to printk() would let you elide 99% of

the non-critical printks pretty easily while only needing to change

the much smaller proportion of the printk()s which are partial

lines.  Furthermore it's pretty easy to grep for the partial-line

printk()s and you can even have it emit warnings when you hit a

partial-line printk() (it doesn't start with "<"[0-9]">") in -mm to

help fix up the last few users and keep people from adding new ones.
Cheers,

Kyle Moffett
-

What about something *really* hardcore ugly like:
#ifdef __FILE__

#undef __FILE__

#define __FILE__ ""

#endif
(or similar preprocessor blecherousness) if you want to *really* shrink

that binary down?

I prefer removing all __FILE__, __FUNCTION__, __LINE__ uses

from printks and defining something that modifies pr_<level>.
Something like:
#define PR_FILE

#define PR_FUNCTION

#define PR_LINE
#if    defined PR_FILE &&  defined PR_FUNCTION &&  defined PR_LINE

#define PR_FMT "(%s:%s:%u) "

#define PR_ARG , __FILE__ , __FUNCTION__ , __LINE__

#elif  defined PR_FILE &&  defined PR_FUNCTION && !defined PR_LINE

#define PR_FMT "(%s:%s) "

#define PR_ARG , __FILE__ , __FUNCTION__

#elif  defined PR_FILE && !defined PR_FUNCTION &&  defined PR_LINE

#define PR_FMT "(%s:%u) "

#define PR_ARG , __FILE__ , __LINE__

#elif  defined PR_FILE && !defined PR_FUNCTION && !defined PR_LINE

#define PR_FMT "(%s) "

#define PR_ARG , __FILE__

#elif !defined PR_FILE &&  defined PR_FUNCTION &&  defined PR_LINE

#define PR_FMT "(%s:%u) "

#define PR_ARG , __FUNCTION__ , __LINE__

#elif !defined PR_FILE &&  defined PR_FUNCTION && !defined PR_LINE

#define PR_FMT "(%s) "

#define PR_ARG , __FUNCTION__

#elif !defined PR_FILE && !defined PR_FUNCTION && defined PR_LINE

#define PR_FMT "(%u) "

#define PR_ARG , __LINE__

#else

#define PR_FMT

#define PR_ARG

#endif
#define pr_info(fmt, arg) printk(KERN_INFO PR_FMT fmt PR_ARG, ##arg)
-

Do we really need another layer of indirection?
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

pr_info and pr_debug do.
It'd make file/function/line cost free for embedded use.
-

I suspect that can be fixed by changing the built system. How can using O=

change the source file path anyway? That seems unnecessary.
It seems to be worse, full pathnames are also used when giving a relative path.

(I'm using O=../obj/).
On the other hand, it doesn't seem to cause that much bloat here:
$ strings vmlinux | grep /home/ |wc

    119     181    6400
CC'ing Sam Ravnborg, perhaps he has some ideas.
Greetings,
Indan
-

Thats a bit unavoidable as the build system works.

__FILE__ is passed the filename supplied as argument to gcc.
Try:

echo "char *s = __FILE__;" > sam.c
gcc -E sam.c

This gives you:

# 1 "sam.c"

# 1 "<built-in>"

# 1 "<command line>"

# 1 "sam.c"

char *s="sam.c";
gcc -E ~/sam.c

This gives you:

# 1 "/home/sam/sam.c"

# 1 "<built-in>"

# 1 "<command line>"

# 1 "/home/sam/sam.c"

char *s="/home/sam/sam.c";
So __FILE__ expand differently depending on the path on

the gcc command line.

I once posted a patch to fix up on this, especialy for BUG_ON and friends.

The solution was to let kbuild generate the filename and to use

this define in the source code.

But a quick grep for __FILE__ in the kernel source made me chicken out.

Simply too much chrunch at that time to justify it.
Googeling a bit I found it here: http://lkml.org/lkml/2006/7/8/22

The better approach would be to use at least the patch inside

the kernel.

This patch should be easy to update to latest kernel if anyone

is up to play with it.
I recall that there was some problems with the path used.

But I cannot remember the details.

Andrew had some inputs from his testing IIRC and google should

be able to tell the full story.
-

Although it's not a big problem, we have the same issue with gcc

warnings and errors displaying the full path with O= builds making the

error messages quite long.
The root is that for O= builds we are feeding gcc the full path

We might perhaps attack it at this point?
The simplest solution that comes into my mind would be to create links

for the source file in the output dir before calling gcc and then give 
cu

Adrian
-- 
       "Is there not promise of rain?" Ling Tan asked suddenly out

        of the darkness. There had been need of rain for many days.

       "Only a promise," Lao Er said.

                                       Pearl S. Buck - Dragon Seed
-

The way I've been building various packages out-of-tree (including the Linux

kernel) is:
cp -sR /path/to/actual/source/tree newdir

cd newdir

configure

make

install

cd ..

rm -rf newdir
The cp -sR creates a new tree of symlinks to the original source, and

everything I've tried so far happily builds in such a directory.
(Yes, this breaks on cygwin.  Ask me if I care.)
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

I'd rather take the opportunity to convert all the printks to

use pr_<level>.  That way, you can pick'n'choose if you want

arbitrary combinations of KERN_<level> compiled in or not.
-

Or:
    if ((1 << level) & CONFIG_PRINTK_MASK)

	actual_printk("<" #level ">" str, __VA_ARGS__);
But it would indeed be nice to have all of pr_<level> (and not only pr_info()

and pr_debug()) anyway.
With kind regards,
Geert Uytterhoeven

Software Architect
Sony Network and Software Technology Center Europe

The Corporate Village

Given that

a) there're plenty of printks without any KERN_* bloat,

b) there're printks that SHOULD NOT have KERN_* bloat,

c) debugging-by-printk method is widely used and this will force

   additional typing, head-scratching  and swear words

d) time wasted on pointless discussions whether some particular

   printk ALERT or CRIT

e) flag day for printk,
Ick.
	Alexey "ignore_loglevel" Dobriyan
--- a/include/linux/kernel.h

+++ b/include/linux/kernel.h

@@ -52,6 +52,7 @@ extern const char linux_proc_banner[];

  */

 #define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#ifdef CONFIG_FOO

 #define	KERN_EMERG	"<0>"	/* system is unusable			*/

 #define	KERN_ALERT	"<1>"	/* action must be taken immediately	*/

 #define	KERN_CRIT	"<2>"	/* critical conditions			*/

@@ -59,6 +60,15 @@ extern const char linux_proc_banner[];

 #define	KERN_WARNING	"<4>"	/* warning conditions			*/

 #define	KERN_NOTICE	"<5>"	/* normal but significant condition	*/

 #define	KERN_INFO	"<6>"	/* informational			*/

+#else

+#define	KERN_EMERG	""

+#define	KERN_ALERT	""

+#define	KERN_CRIT	""

+#define	KERN_ERR	""

+#define	KERN_WARNING	""

+#define	KERN_NOTICE	""

+#define	KERN_INFO	""

+#endif

 #define	KERN_DEBUG	"<7>"	/* debug-level messages			*/
 extern int console_printk[];

-

So define a level 0 that doesn't prepend any level to the string, and have the

macro filter that out at the same default level it counts as now.

(KERN_INFO, I think?)  The tests are all on contants which should resolve at

compile time and the dead code eliminator should zap it, even if the macro 
Let me get this straight: you're objecting to actually making the printk

levels useful enough that developers start to care what they're set to,

because then they might be motivated to want some of them changed?
Make it useful, people might care, thus they might talk about it...
That's the main reason I haven't played with it so far, although it would be

easy to define a new symbol (dprintk or some such, although I note several 
*Shrug*.
My problem is that switching off printk is the single biggest bloat cutter in

the kernel, yet it makes the resulting system very hard to support.  It 
But ignore_loglevel doesn't decrease the size of the _binary_.  That's what

we're talking about here with the -tiny tree.  Embedded developers want to

squeeze more code onto smaller flash/rom chips.  Setting ignore_loglevel does

prevent these messages from ever being emitted, but they're still in the

kernel image as dead weight.  It saves noise but doesn't save _space_.
I'm proposing allowing an ignore_loglevel to remove the unused messages at

compile time so they don't take up space.  Doing that requires the levels to

be integers so they can be compared with < or >, and the remaining changes

follow logically.  (To me, anyway...)
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

What about getting even more hard core? 
Use compiler tricks to remove ALL the static printk string from the

kernel and replace the printk with something that outputs an decimal

index followed by tuples, of zero to N, hex-strings on a single line.

Then have the syslogd or some other utility take this cryptic output and

convolve it with a table (created at compile time) to re-create what

would have been dumped to the sys-log ring buffer.  This way you strip

out most of the static text from the kernel and yet can still re-create

the kernlog output.
At least as a post processing operation....
Is this an old idea?  I'm guessing this has been at least proposed

before....
--mgross
-

* Thu, 20 Sep 2007 15:15:47 -0700

* X-MimeOLE: Produced By Microsoft Exchange V6.5
Not all, but critical info, that must exist in human-readable form of
Sure, but a little problem is, that many kernel developers do C (mostly)

and Perl (occasionally), i.e. very few do non-trivial userspace (even

userspace do too much C and Perl sometimes [:
Seriously. When in the Windows there are only messages like:
    "Error (Code:0x00002012)".
In the Linux... well, embedded targets, this can be turned in a very

efficient thing by means of userspace. On other setups this can be nice

and pleasant one, with yet another L10N project, recently promoted by

README translations. But,,, see problem above.

____

-

I disagree.  For a production product the you want minimal information

to reduce the communication bandwidth required between the remote

customer and the support organization.
In fact there is a good argument that you don't what the remote customer

to know enough to start guessing.  In the support stage of a products

life cycle you really don't want guessing or fudging of information

based on guessing.  Keeping the output cryptic and short will avoid

these things.  (That really does happen and it cost a lot in support
Oh poo.  Kernel programmers can use user mode tools and write them too,

and I *know* even I could write such a post processing program to take a

somewhat compressed and cryptic output and generate what would have been
Now it's been ~8 years since I did any serious windows work, but if I

recall correctly ALL THE FRICKING TIME!!! When was the last time you've
From you comments I'm not sure you are for or against this idea.
--mgross

-

Don't use Linux then.  Open source is a horrible fit for the way you think.
I'm sympathetic to "shrink the binary size" arguments.  I'm not really

sympathic to "keep the customer in the dark intentionally" arguments, whether

the justification is "because they're stupid", "to increase dependency on our 
I believe he was holding it up as a bad example, and definitely not something

we want to emulate.
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

On Fri, Sep 21, 2007 at 04:15:39PM -0500, Rob Landley wrote:
I tried to show, that keeping users in compete information vacuum is a

bad thing. Even without sources, _configuration_ makes another area of

mis-working and bugs, usually addressed by reinstalling.
That may be bad example, because here talk is about developers and

testers, who are not just ordinary users. And by applying Torvalds's Law,

all users are such in some degree. That's why {1} in your reply, Rob,

makes perfect sense.
If Mark have a bad experience with lusers only, then i just can say: what

a pity! AFAIK nobody can read somebody's plain-bin OOP output.
Anyway, anything must be opted by config options, even schedulers. But

maintenance and flame wars rule otherwise :).
What i can propose is form of binary-only "printk", where all info:

diagnostic, error, bug, statistics messages (in not debugging

environment, of course), is just fed right to output buffer (see,

pa, no kmallocs). Info itself must have structured content, that makes it

easy to extract and locate human-readable representation of both message

and data.
This doesn't address loglevels, though.
Implementation (seems) as easy as feeding output to `od` to have

unambiguous form of various troublesome bytes, like "0x00" and "0x0A",
Structuring, who is printing, i.e. arch code, fs, driver whatever, must be

agreed:
*  Profiles[0]: originator's ID of a message is a byte (or word, or double word)

   0x01 - arch, 0x02 - fs, 0x03 - net, 0x04 - hw drivers, etc.
*  Data itself can be sent in form of [0]
[0] Banana -- extendable protocol for sending and receiving s-expressions

    http://twistedmatrix.com/projects/core/documentation/specifications/bana...
and having shell script with functions, that have names that correspond

to actual structured content:

_*_

olecom@flower:/tmp$ sh banana.sh < banana.c >bb

olecom@flower:/tmp$ sh -c '. ./bb ; _07080'

start

olecom@flower:/tmp$ sh -c '. ./bb ; _07081'

ti_startup - product 0000, num ...
[ message continues ]

think.
I'll do what I like, thank you.  I'll continue to use Linux, I think
You are now talking religion at this point.  Do you have a technical or

even experience based point to make?  
I have experience that has shown that providing too much information in

a production product can is confusing, harmful and costly when dealing

with consumers.  Your opinions won't change that.
I proposed a mechanism for keeping all the printk data and saving space

buy doing some table based compressions that has the side effect of

making the syslog not human readable.  You proposed a mechanism for

no-oping out complete log-levels.   
Which way hides more from the user?  No-oping the log-levels is the
There is a time and place for many things.  Even error codes. 
--mgross

-

How about compiler tricks to compress the static printk strings?

These could be expanded at runtime to use as the format.
Timothy Miller suggested something similar awhile ago.
-

You would have to hold the text table (compressed) in memory to do this

-

> > I think that this idea is not worth it.
It's not such a big downside IMHO.  You can support a kernel without

printk.  Need to debug the kernel without printk?  Use a JTAG

debugger...
If you have a system that actually configures out printk's, chances

are you don't have storage and output mechanisms to do much with the

messages anyway.  Think embedded _products_ here.  Sure the

development boards have serial, ethernet, and all that jazz but tens

of millions of ARM based gadgets don't.

-

I don't actually own a jtag.  (I do use qemu's gdb support to debug the target 
I wonder if that "Monsoon" gadget does?  (Sorry, just on my mind today...)
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

Just to clarify, which bloat are you concerned about?

I presume source code bloat (but maybe you mean

message size bloat, or object code bloat)?
 -- Tim
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-

Users of ignore_loglevel still has "<[0-7]>" prefixes in kernel image,

yes. On source level, too -- as someone who never saw value in them,

KERN_* are just needless characters, making code harder to read.

-

Yeah, I bumped into this when I implemented dmesg for busybox.  (Had to parse

and filter it out.)  That said, userspace kind of expects it now...
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

On Wed, 19 Sep 2007 11:03:09 -0700
I volunteer!  Send patches to me, cc linux-kernel and celinuv-dev.
Seriously, putting this stuff into some private patch collection should

be a complete last resort - you should only do this with patches which

you (and the rest of us) agree have no hope of ever getting into mainline.

-

History!
<computer historian hat on>
The -tiny tree started out as a separate patch kit of Matt Mackall's, which he

stopped updating circa 2.6.14 because he didn't think keeping them out of

tree was helping attract other developers, nor was it helping to get them

inline.  He decided to focus on pushing the existing patches into mainline,

and stop maintaining the out of tree patcheset for new releases.  His last

post on the subject (to the linux-tiny mailing list) was a year ago:

http://selenic.com/pipermail/linux-tiny/2006-March/000314.html
But what happened is that most of the abandoned patches stopped applying to

new kernels yet still weren't available in mainline a year later, so Tim and

Michael have stepped in to revive the -tiny tree.  (Tim talked about this a

bit at the CELF BOF at OLS, which is more acronyms than should really show up

immediately after one another in any confersation, FYI.)
So yay new tree.  Tried without it, didn't work.  Broken up to make merging

easier, but mainline will probably never _fully_ catch up, any more than

it'll catch up with any of the other special-interest development trees.

Making -tiny an .hg tree would be really really nice, though... :)
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

OK, I'll try to accelerate the effort to send these to you.

We'll still need some kind of bucket for the patches that

don't apply to recent kernels, but which no one has yet

had time to bring up-to-date (or evaluate for permanent

dismissal).  And dribbling them out, fixing them up,

responding to issues - all take time that I can't

commit to personally for the next week or so.

I'll let Michael respond whether he can get to this

sooner rather than later, as planned.
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-

Andrew, you're completely right... The patches should all aim at being

included into mainline or die.
I'm finishing a sequence of crazy weeks and I will have time to send you

patches one by one next week, starting with the easiest ones.
Thanks for your support.
Cheers,
Michael.
--

Michael Opdenacker

http://free-electrons.com

+33 621 604 642
-

Well thats good news.  In response to the comments made about testing

the impact of these patches on big-iron I was going to suggest we ask

Andrew to include your patch set in -mm so that it firstly gets at least

compiled on big-iron, and secondly so we could think about how to test

with some of the options enabled on big-iron.
Knowing nothing about these options, from a test perspective it would

be nice if we were able to simply enable "the lot" so we can do "normal"

-mm runs and "tiny" -mm runs without any manual intervention?
-apw

-

I agree completely.
I have been thinking for a while about how to make a "monster switch"

(the kind they always seem to have in Frankenstein movies) that

switches a whole bunch of settings at once.  We currently have methods

in the kernel for:

 * default (or recommended) config for a particular platform

 * all yes - to build as much as possible

 * all no - to build as little as possible
The problem with "allno" is that it rarely produces a usable

kernel.
There are three possible approaches that I can think of:

 1) use a tool to start from default and turn off options

 to make a small (but still usable) config

   * I have a tool to do this now as part of my automated test

   I haven't published it, but I can if anyone's interested.

 2) use the kconfig dependency system to disable stuff automatically

 if someone chooses the "make_it_small" option.

 3) create defconfig_small files for the platforms that care about

 size
3) is easiest to implement at first.  It's trivial to make a new

defconfig, and we could easily come up with a convention for them.

However, they would be a pain to maintain (this would essentially

double the defconfig maintenance), and you'd have to convince

people that it's worth carrying these in the mainline tree.
I haven't looked at 2), so I'm not sure exactly what would be

involved.  I'm not sure if you can centralize all the dependency

information in the "make_it_small" option, or if you'd have

to spread it out to the related configs.  I'm not even sure which

arrangement of the info would be the easiest to maintain.  Would

it be best to have a single size-conscious person maintain the

dependencies, or better for config authors to maintain this info

in parallel?
Anyway, those are just some thoughts on the subject.

Feedback on an acceptable solution would be welcome.

 -- Tim
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-

Beyond that, allno doesn't come close to switching everything off.
1) You have to _enable_ CONFIG_EMBEDDED in order to go into that menu and

switch _off_ the stuff in there.
2) The stuff CONFIG_EMBEDDED reveals isn't all in that menu.  CONFIG_BLOCK is

at the top level menu.  CONFIG_VT and CONFIG_UNIX98_PTYS are buried down

under device drivers->character devices, and there's more sprinkled all over.

You have to track it all down and switch it off to get an _actual_

allnoconfig kernel.
(I cut the bit where you reinvent miniconfig.  People keep doing this.  I dig

it up and resubmit it every year or so, so Roman Zippel can shoot it down

again.  Meanwhile, not only is Firmware Linux happily using it, but I even

wrote more documentation at

http://landley.net/code/firmware/new_platform.html although you have to

scroll down a bit to get to the stuff about miniconfig...)
Rob

--

"One of my most productive days was throwing away 1000 lines of code."

  - Ken Thompson.

-

DetaolB (see signature) is also using miniconfig, which is good

existing code. It can actually be pushed further by storing a

/proc/miniconfig.gz rather than the full /proc/config.gz
You can find patches on how to perform this inside the iso of DetaolB v0.6
--

Christian

--

http://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !

-

I use it for daily build/boot/run-some-number-like-30-tests kernel testing.
---

~Randy

*** Remember to use Documentation/SubmitChecklist when testing your code ***

-

Not sure what the point is of a separate patchkit. If it's a reasonable

patch it should just be put into mainline. And hopefully all the patches

will be reasonable.
-Andi

-

I don't know the detailed history, but my understanding is that

for some of these, mainline attempts were made in the past.
The patchkit gives a place for things to live while they are out

of mainline, and still have multiple people use and work on them.

Optimally the duration of being out-of-mainline would be short, but

my experience is that sometimes what an embedded developer considers

reasonable to hack off the kernel is not considered so reasonable by

other developers (even with config options).  Also, sometimes it

takes a while for a patch to mature to the point where it is

acceptable for general use, while still being usable for

special-purpose projects.

 -- Tim
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-

Is anybody working on testing that the patchkit "does no harm" for bigger

boxes (laptops, desktops, servers)?  That would probably be helpful info

when pieces are pushed towards mainline....
(Was just looking at the patch details, there's at least a few that look

interesting for my purposes even though I'm not an embedded developer)...

Not to my knowledge.  Most of the things it provides are

only activated by config options.  So my sense is that just

applying the patches should be harmless.  But we'll need some

runtime testing to see what affect some of these have on

big iron.
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-

Tim is right, Linux-Tiny features are only activated by configuration

options.
However, that doesn't prevent us from proposing size reduction changes

impacting all kernel users, in cases a configuration option doesn't make

sense. Extensive testing in the -mm tree will then be required.
Cheers,
    Michael.
--

Michael Opdenacker

http://free-electrons.com

+33 621 604 642
-

congratulations to the new maintainer!
nice to see this subset of patches being revived.
;-)
--

Christian

--

http://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !

-

Will there be a separate git for testing? Is the idea to still keep

moving patches upstream?
  Luis

-

Right now we're publishing a quilt-able tarball.

Instructions for applying this are on the

http://elinux.org/Linux_Tiny page.
Note that some broken patches are kept around

in the patches/tiny directory, but commented

out in the series file, in case someone
Yes, very much so.  These patches have been

showing up in CE products, and this is part

of an overall effort to dust them off, test

them, and move them upstream.

 -- Tim
=============================

Tim Bird

Architecture Group Chair, CE Linux Forum

Senior Staff Engineer, Sony Corporation of America

=============================
-