Following Andrew Morton's recent comment, "this just isn't working any more," Miles Lane asked, "what can be done to reduce the huge number of build fixes required to release an MM tree?" Andrew jokingly replied, "my mind turns to cattle prods." Regarding the suggestion that he could publicly list the offenders he quipped, "I could name names, but it would look like 'grep @ MAINTAINERS' ;))" He continued to say, "I don't think much can be done about it, really," going on to explain:
"See, what I do is to merge probably hundreds of patches into the -mm-only part of the tree and then, after a few days, get down and compile-test it all, then fix it, then runtime test it all, then fix that. Because it is vastly more efficient to do all this work against hundreds of patches than it is to do it against one patch at a time, no?
"And guess what? All the other maintainers do the same thing: someone sends you a patch, it looks good, so you commit it. After you've committed a decent batch of patches, get in there and test it all. Problem is, I often will get in there and do all that testing before the subsystem-tree owner has done his testing."
From: Miles Lane <miles.lane@...>
Subject: What can be done to reduce the huge number of build fixes required to release an MM tree?
Date: Sep 18, 8:23 am 2007
Hello Andrew and all,
What can be done to reduce the huge number of build fixes required to
release an MM tree?
Perhaps it would be helpful if you identified specific individuals who
send you patches that break the build. If necessary, we could keep a
running total. The main thought is that maybe we need to identify who
regularly sends troublesome patches. There are three metrics that
might be good to know: 1) Who sends patches that are regularly
broken, but sends patches rarely? 2) Who sends patches all the time,
and once in while causes trouble? And 3) Who is causing the most
cumulative trouble?
Is there any possibility that we ought to refuse patches from people
who break the build too often?
Another area that might be helpful might be how the patches break
things. Are there major groupings of errors that developers could be
educated about?
I am sorry you are having to work so hard to do your job, Andrew.
Miles
-
From: Andrew Morton <akpm@...>
Subject: Re: What can be done to reduce the huge number of build fixes required to release an MM tree?
Date: Sep 18, 3:48 pm 2007
On Tue, 18 Sep 2007 08:23:38 -0400
"Miles Lane" <miles.lane@gmail.com> wrote:
> Hello Andrew and all,
>
> What can be done to reduce the huge number of build fixes required to
> release an MM tree?
My mind turns to cattle prods.
> Perhaps it would be helpful if you identified specific individuals who
> send you patches that break the build. If necessary, we could keep a
> running total. The main thought is that maybe we need to identify who
> regularly sends troublesome patches. There are three metrics that
> might be good to know: 1) Who sends patches that are regularly
> broken, but sends patches rarely? 2) Who sends patches all the time,
> and once in while causes trouble? And 3) Who is causing the most
> cumulative trouble?
>
> Is there any possibility that we ought to refuse patches from people
> who break the build too often?
>
> Another area that might be helpful might be how the patches break
> things. Are there major groupings of errors that developers could be
> educated about?
>
>
> I am sorry you are having to work so hard to do your job, Andrew.
>
I don't think much can be done about it, really.
See, what I do is to merge probably hundreds of patches into the -mm-only
part of the tree and then, after a few days, get down and compile-test it
all, then fix it, then runtime test it all, then fix that. Because it is
vastly more efficient to do all this work against hundreds of patches than
it is to do it against one patch at a time, no?
And guess what? All the other maintainers do the same thing: someone sends
you a patch, it looks good, so you commit it. After you've committed a
decent batch of patches, get in there and test it all.
Problem is, I often will get in there and do all that testing before the
subsystem-tree owner has done his testing.
The only way in which my problem can get fixed is if all the subsystem tree
maintainers were to do a full set of compile-testing and runtime testing
before committing each change (impossibly expensive) or if they were to run
two trees: a raw/untested one, and a separate cooked/tested tree. I then
pull the cooked/tested one. The latter approach has quite a lot of
overhead for everyone as well.
But really, it's not worth sweating over the build errors: they're almost
always trivial to fix, and they are of course immediately apparent.
So don't worry about the build errors - they're a trivial nuisance. It's
the runtime errors which are much, much more serious and are much more
expensive to fix and which affect our users much more seriously.
(otoh, quite a number of the build errors are really really silly ones,
where someone couldn't be assed cooking up a decent grep regexp or
didn't even compile-test the change with _any_ config. I could name
names, but it would look like `grep @ MAINTAINERS' ;))
-
Scary!
If this is the Linux development model and the way things are patched/tested and hundreds of patches from everyone and their grandmother's dog, along with *experimental* and sloppy, poorly written/documented code in the kernel, then I'm truly scared to even use Linux anymore at home - let alone in a production environment!
No better than MS Windows.
Yes, IF.
Yes, IF.
Sigh
Ok, I'll bite.
You're staring into the real working kitchen here, sorry it's not all perfectly choreographed pretty chefs in tutu's dancing around each other en-pointe for you, but there's real work to do.
Beware, the troll/FUD army
Beware, the troll/FUD army is attacking again (hint: look for similarities in the first three root-level comments)
This is the -mm kernel tree whose only purpose is to carry large patches that are not yet ready for the mainline. These patches have often gotten very little review and are still under development, so their side-effects can be unclear. It would be surprising if they didn't cause any problems.
Having used -mm kernels for about a year in the past, I was certainly impressed with the quality. Apart from a couple of problems which stopped the kernel from booting up, I really had no significant issues. I stopped using it because keeping up with the pace of releases is quite challenging if you're not a kernel develoepr (for every -rc that Linus's releases, there are often several -mm releases). Another frequent problem was binary drivers whose wrapper code had to be changed to get them compiling.
Linux is a Design Mistake!
I think this is the long awaited problem and flaw outburst of the Linux Kernel. This is an unavoidable consequence of the monolithic kernel design. It's just to complex to track everything or get at least an overview. Everything changes daily and all knowledge becomes outdated and useless after a small period of time.
It somehow reminds me of the book "1984", they change all the news, but if they make a mistake, their system will "crash". The same is true for the Linux Kernel and I think everyone stumbles on such mistakes frequently.
But this whole debate reflects the incompetence of the Kernel developers. They chose the wrong design model and dictators like Linus Torvalds are actively defending their mistake and claim it's superior. This is really self-defeating, but it seems that a lot of people like it. Strange, isn't it?
yes i also ask myself if this could be really Intelligent Design
Your comment appears to me in the same intellectual level as the whole ID discussion.
Some fish for you ><=>.
PS: I know i shouldn't but it was so tempting, sorry folks.
Well, guy or girl, you are
Well, guy or girl, you are clearly lacking a lot of communication skills, hmm, all of them ;-)
OTOH, you make two statements that are useful to be discussed.
1/
2/
True or false? Or is it more a kind of grayish?
Subsystem leads are trusted?
This is really more of a failure of configuration management than overall design. Sorry microkernel fanboys. How is it that all of these subsystem leads are "trusted" yet are able to submit patches that aren't tested and don't compile in the integration tree? Everywhere I have ever worked uses the "you break it, you bought it" method of development. You quickly learn not to commit crap to the source tree. This GIT patch system that Lignus is so fond of doesn't help matters.