IDE (file changing/edit, flexible configure, building on-the-go): Linux kernel kconfig and kbuild alternatives.

Submitted by olecom
on May 4, 2008 - 9:10am

Important stuff. But it needs much more work, than linux kernel has and all useless GUI stuff does.

1) message after coding one smaller replacement of `test` in `dash`
2) message after yesterday's exercises with input, adds UI part.
3) side note about recurcive `make` and useless `make` itself.

Kernel-userspace coders' gap is big. Yet where are game-coders. This task for them, and was done decades ago in closed game-coding cycles.

From olecom Mon Sep  3 23:48:59 2007
Date: Mon, 3 Sep 2007 23:48:59 +0200
Subject: Re: kconfig/kbuild rewite (Re: What's up with CONFIG_BLK_DEV?)
Message-ID: <20070903214859.GW475@flower.upol.cz>

On Sun, Sep 02, 2007 at 01:51:50PM +0200, Sam Ravnborg wrote:
[]
> Then as now you have not yet expalined what you are trying to do.
> Nevertheless I look forward for a minmal set of patches that improve
> whatever you are working with.

Yes, because it's LKML, that wants not-hand-waving stuff in first
place. But recent kevent, scheduler stuff shows
inflexibility/agendas, though.

That examples were bad WRT kernel production infrastructure. Even then i
doubt, i must waste everyone's time with details, only goals.

> As for Kconfig the low hanging fruits are not in the tools but in the
> structure of the Kconfig files. There are a lot that can be improved
> with a decent effort but nobody has stepped up doing so.
> The tools could be better too but if the root problem is the structure
> of the Kconfig files this is where we should focus and not on the tools.

In my view this all interconnected. Designing flexible and easy
configuration system yields dramatic changes to the build one as well.

* profiles non/debug, non/production

* per file, per algorithm tuning
  * efficiency
    * choosing structure sizes
    * selecting fast/slow paths
    * per case choosing need/dead code
    * various parameters
  * optimization
    * per compiler/version
      * option profiles
      * feature/warning sets
    * linker
      * is there anything alternative?

* distributed development
  * open possibility to work in any part of the tree
    * making changes and quickly having
      * config (dependency, etc.) set/UI ready
      * per profile/option test builds
        (e.g. making return->goto or loop change and quickly getting -O0,
        -O2, -Os images; check size; have userspace testing skeleton -> have
	runtime test)
    * integration with quilt-like source/patch managers ``here''
  * allow per architecture development
    * small source tree
    * developer's profiles, that will have exact feature/tuning/build
      config options results for everybody within given source tree version
      (for easy testing, but not "send me your .config; what binutils?..")

* base set of tools to have easy to configure alternatives
  * shell
    to use basic POSIX (plus accepted, not NIH like in bash) features
    (i have some examples; unfortunately even basic set behaves
     differently and buggy)
  * make
    stat() wrapper executing shell everywhere; of course there are
    some features, but anyway, interface for it and the like is needed
  * perl/python/ruby
    establish text processing rules
  * coreutils/busybox/etc
    non is perfect, having replacement mechanism allows faster debug
    and enhancement of their own development and testing

* UI
  (maybe next time)
  
  Only one thing. I don't have time and will to study all that
  ncurses/slang/qt/gtk/AJAX/whatever stuff. I wanted to do basic terminal
  or text/stream editor friendly user interface. As for the former, i
  just upset about software capabilities of the todays terminal
  emulators. I'm fine with exchanging escape sequences, but all that
  inherited TEKTRONIX 4010, APL, HP2645, Microterm ACT-IV, Ann Arbor
  4080, LSI ADM-3a (man terminfo) legacy without even a hint of progress
  last 20 years is just dead.
  
  I likely to end up with shell script generation, that will be
  available for everybody who knows shell and have ordinary text editor.

  autoconf/configure inside out? Maybe, but at least from the new sheet of
  paper, with good background in history and basic text processing tools.

Just in case anybody cares about how ugly modern software development is
(INA software industry dude, and may be just crazy, of course). Well,
recent Rusty's gig may give a clue, how things may look different.

> For Kbuild I fail to see anything that demand a rewrite from a structure
> view of the Kbuild files.
> The Kbuild internal stuff is antoehr story - here a rewrite to a saner
> language then GNU make syntax could improve hackability a lot.
____

second: text user interface for multidimentional walk and config

olecom
on
May 4, 2008 - 9:14am
From olecom Sun May  4 17:01:30 2008
Date: Sun, 4 May 2008 17:01:30 +0200
Subject: Re: kconfig - a suggestion how to fix the select issue
Message-ID: <20080504150130.GB24008@flower.upol.cz>

David Collier-Brown @ Sun, May 04, 2008 at 08:55:12AM -0400:
[]
> I speculate that having two ways to express a dependency,
> and the addtition of visibility control makes the
> dependency tree-walk into a problem which is no longer
> solvable in trivial logic. That in turn makes my head
> explode (;-))

In short:
mixing all together and applying resolving and UI heuristics on top.

> I wonder if one could simplify back into a flat set of
> selections without visibility rules and a backwards-
> chaining "you need to select these too" message emitter,
> and if that would be worthwhile.

No need to hide anything and invent complexity. Flexible and
human-friendly interface. (Batch things like def_condig are done by
humans, randconfig -- useless if there's no heuristics, allyes - simple.)

Just UI part for now. Automation of selection can be applied only, after
one can see whole picture. Say, you have [ UCcamers ] which depends on
USB, Video, etc.

+-- Navigation ---------------------------------------------------+
|  ...            [ arch     ]      [ atm ]     [ w-less speaker] |
|  [ kernel ] --> [ drivers  ]  --> [ fun ] --> [ camerass      ] |
|  [ klibc  ]     [ file sys ]      [ mtd ]     [ manual DMA    ] |
|  ...            ...               ...         ...               |
+-----------------------------------------------------------------+
+--     needs     ------+  +-- configure -----+  +--  options  ---+
| ++Universal S bus++++ |  |  ...             |  | build: y [m] n |
| --Video capture---    |<-+-[* UCcamera    ]-+->| RGB support y  |
| --sysfs kernel for u- |  |  * LPTcamera     |  | YUV support y  |
|                       |  |  * RS232camera   |  | HSB support y  |
+-----------------------+  +------------------+  +----------------+
+-- command line/text editor -- olecom@kit[configure] ------------+
$ build = m \n
$ go <

+--  walk tags  ---------+ +------------- Description ------------+
| intro, kernel, drivers,| | USB camera with foo bar features     |
| fun, [camerass]        | | with dual-core toaster and coffee    |
|                        | | machine inside                       |
|                        | |                                      |
|                        | | Options:                             |
|                        | | * RGB enables RGB                    |
|                        | | ....                                 |
+------------------------+ +--------------------------------------+

So, after 'go <' command (visually or command-lineually) we will be in
[--Video capture--] which isn't configured/enabled yet (due to --)

Now,

+-- Navigation ---------------------------------------------------+
|  ...            [ arch     ]      [ atm ]     [ w-less speaker] |
|  [ kernel ] --> [ drivers  ]  --> [*fun*] --> [ camerass      ] |
|  [ klibc  ]     [ file sys ]      [ mtd ]     [ manual DMA    ] |
|  ...            ...               ...         ...               |
+-----------------------------------------------------------------+
+--     needs     ------+  +-- configure -----+  +--  options  ---+
| ++Universal S bus++++ |  |  ...             |  | build: y [m] n |
| [-Video capture--]    |<-+-[* UCcamera    ]-+->| RGB support y  |
| --sysfs kernel for u- |  |  * LPTcamera     |  | YUV support y  |
|                       |  |  * RS232camera   |  | HSB support y  |
+-----------------------+  +------------------+  +----------------+
+-- command line/text editor -- olecom@kit[ needs   ] ------------+
$ build = m \n
$ go < \n
$ we are here, hitting enter on needs/video
+--  walk tags  ---------+ +------------- Description ------------+
| intro, kernel, drivers,| | Video processing for everyone        |
| fun, cameras, [videoP] | | all is supported all works with no   |
|                        | | OOP and shaman drums                 |
|                        | |                                      |
|                        | | Inside MMA:                          |
|                        | | * conf: DEBUG                        |
|                        | | * build: chip1...                    |
+------------------------+ +--------------------------------------+

we will have:

+-- Navigation ---------------------------------------------------+
|  [ arch     ]    [ mtd   ]    [ radio ]    [ bt8xx    ]         |
|  [ drivers  ] -> [ media ] -> [*video*] -> [ cpia2    ]         |
|  [ file sys ]    [ ps3   ]    [ common]    [ usbvideo ]         |
|  ...            ...               ...         ...               |
+-----------------------------------------------------------------+
+--     needs     ------+  +-- configure -----+  +--  options  ---+
| --V4L1--              |  |  ...             |  | menu:          |
| --V4L2--              |<-+-[*Video capture]-+->| * DEBUG_FOO    |
| --V4W7--              |  |  * Video decode  |  | * bt8xx        |
|                       |  |  * Video improve |  | ...            |
+-----------------------+  +------------------+  +----------------+
+-- command line/text editor -- olecom@kit[configure] ------------+
$ go < \n
$ we are here, hitting enter on needs/video
$
+--  walk tags  ---------+ +------------- Description ------------+
| intro, kernel, drivers,| | Video processing for everyone        |
| fun, cameras, [video]  | | all is supported all works with no   |
|                        | | OOP and shaman drums                 |
|                        | |                                      |
|                        | | Inside MMA:                          |
|                        | | * this is menu                       |
|                        | |                                      |
+------------------------+ +--------------------------------------+

== [ rant ] ==

Can some of those PS3 or KDE teams design proper "Konfigure Linux"
game for kernel hackers? I try to do this with TERM=linux, `sh`, and
`sed`. Patching paper work isn't for such tasks.

On collecting CONFIG_ info by gcc in sources, thus generating symbol
dependency automatically, was told to shut up. Let's see what will
happen next. Don't forget to use cultural skills, imagination and
creativity.
--
sed 'sed && sh + olecom = love' << ''
-o--=O`C
 #oo'L O
<___=E M

do they edit sources with `cat`?

olecom
on
May 4, 2008 - 12:54pm

Do they edit with cat >source.c? Oh crap.

Date: Sun, 4 May 2008 21:30:32 +0200
From: Sam Ravnborg
Subject: Re: private Re: kconfig - a suggestion how to fix the select issue

On Sun, May 04, 2008 at 07:28:53PM +0200, Oleg Verych wrote:
> Sam, hope you saw that mail with UI part. It complements what i've wrote
> to you:
> 
> Date: Mon, 3 Sep 2007 23:48:59 +0200
> To: Sam Ravnborg 
> Cc: Rob Landley rob@landley, Jan Engelhardt jengelh@computergmbh,  linux-kernel@vger
> Subject: Re: kconfig/kbuild rewite (Re: What's up with CONFIG_BLK_DEV?)
> Message-ID: <20070903214859.GW475@flower.upol.cz>
> 	
> And maybe you can understand my rants about simple-minded kconfig
> interface and simple-cloning like `mc` in userspace programming. I'm not
> a game, flash or web 2.0 or any other gui designer/programmer, but i can
> see how inflexible configuring of "the biggest foss project is".
> 
> Are we going to be more and more stupid, doing dumb job with over-complex
> high-tech techniques? Oh, gee...

The only thing I see is that you want to change everything
whre I (and most of the community) want to go in 
smaller steps should we decide to change something.

So I did not see you UI proposals as usefull here - sorry!

	Sam

`make` is not needed.

olecom
on
May 4, 2008 - 9:21am

`test` -nt -ot are quite OK instead. More work on collecting dependency/configuring and flexible build is needed.

From: Oleg Verych
Newsgroups: gmane.linux.kbuild.devel
Subject: Re: Non-recursive kbuild
Date: Thu, 25 Oct 2007 05:39:16 +0200
Organization: Palacky University in Olomouc, experimental physics department
Message-ID: <20071025033916.GE22435@flower.upol.cz>
NNTP-Posting-Date: Thu, 25 Oct 2007 03:24:44 +0000 (UTC)

* Wed, Oct 24, 2007 at 06:47:20PM +0200, Vegard Nossum:

> I have an idea on how to implement kbuild using non-recursive make,
> which should be both more efficient and safer than the current recursive
> make.

Well, i have ideas too. But they do not consider `make` as any kind of
"efficient" and "safer" thing, whatever ad hocs someone going to stick to
it.

The most referred paper about recursive make states:

   How can it be possible that we have been misus-
   ing make for 20 years? How can it be possible
   that behavior previously ascribed to make's limi-
   tations is in fact a result of misusing it?

   The author only started thinking about the ideas
   presented in this paper when faced with a number
   of ugly build problems on utterly different
   projects, but with common symptoms. By step-
   ping back from the individual projects, and
   closely examining the thing they had in common,
   make, it became possible to see the larger pattern.

   Most of us are too caught up in the minutiae of
   just getting the rotten build to work that we don't
   have time to spare for the big picture. Especially
   when the item in question "obviously" works[...]


But after reading this earlier in the paper,


   It must be emphasized that this paper does not
   suggest that make itself is the problem. This
   paper is working from the premise that make does
   not have a bug, that make does not have a design
   flaw. The problem is not in make at all, but rather
   in the input given to make - the way make is
   being used.


i just am going to see "a big picture", which is without `make`. It
doesn't matter, if `make` was misused for 20 years. What is matter, that
for 9 years, after this topic was brought to LKML by the paper author,
nothing has changed in the linux build system, WRT this problematic.

Maybe someone is going to get rid of gotos, or doing whatever other
"crap" CS professors think? Only reality shows, what results of
any kind of hand wavings or implementation are. For now this are:

* gotos are good in the system programming, when used appropriately
  (it seems, that famous kernel developer doesn't get it right
   sometimes (anytime?) <20071007205954.4053C13E06@wotan.suse.de>);

  the main feature request for C lang lawyers is: "labels should have
  addresses, programmer can read and manipulate."

* micro kernels -- no micro kernels (see the WWWeb :)
____
-
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.