From: Linus Torvalds <torvalds@...>
Subject: Re: Announce: Linux-next (Or Andrew's dream :-))
Date: Feb 12, 3:55 pm 2008

On Tue, 12 Feb 2008, Greg KH wrote:
> 
> > That's the point.
> 
> Not it isn't.  To quote you a number of years ago:
> 	"Linux is evolution, not intelligent design"

Umm. Have you read a lot of books on evolution?

It doesn't sound like you have.

The fact is, evolution often does odd (and "suboptimal") things exactly 
because it does incremental changes that DO NOT BREAK at any point.

The examples are legion. The mammalian eye has the retina "backwards", 
with the blind spot appearing because the fundmanetal infrastructure (the 
optical nerves) actually being in *front* of the light sensor and needing 
a hole in the retina to get the information (and blood flow) to go to the 
brain!

In other words, exactly *because* evolution requires "bisectability" (any 
non-viable point in between is a dead end by definition) and does things 
incrementally, it doesn't do big flips. It fixes the problems on an 
incremental scale both when it comes to the details and when it comes to 
both "details" (actual protein-coding genes that code directly for some 
expression) and "infrastructure" (homeobox and non-coding genes).

So quite frankly, you're the "intelligent designer" here. You're the one 
who seems to claim that we need those leaps of faith and wild jumps.

> Oh, it's been painful at times, but they are, overall, very rare.

No, overall, they have *not* been rare lately. We've had them all over. 
And not just the one introduced by you.

> If you look at the rate of change we are currently running at, it's
> amazing that we do not get _more_ of these kinds of problems.

I don't think that's a valid argument.

Sure, we have lots of changes, but 99.9% of them have no cross-subsystem 
effect what-so-ever.

> > And simply avoiding cross-subsystem API changes unless there is a major 
> > *MAJOR* reason for them is the obvious thing to do. Simply face the fact 
> > that even in open source there are major reasons to stay with an old 
> > interface even if it's not optimal.
> 
> I strongly disagree here.  We lived with that kset/ktype crap for years,
> and I finally broke down and cleaned it up, simplifying things, removing
> code, making the kernel smaller, leaner, and easier for others to change
> and use in the future.  With your statement, such a change should have
> never taken place as it what we had at the time was "not optimal", but
> good enough to live with.

You didn't listen at all.

I said that the threshold should be high, not that it should be 
impossible. I also said that we should strive for making it unnecessary to 
have the painful total synchronization points.

The fact is, we *have* been able to do things like this gradually and 
well, without introducing breakage. Take the VM changes, for example: 
those were pretty damn fundamental, where we've changed the calling 
convention totally for fault handling.

But that thing was done without at any point really seriously breaking 
code. It involved adding the new interface, and letting the old one live 
in parallel. 

The last remnant of the old "nopage()" interface still exists, but I think 
right now it's only used by DRM. 

Did it require the drivers to be updated? Yes. But it did NOT require the 
total synchronization, because it still worked with the old interface.

> But they do happen about once or twice a kernel release, just by virtue
> of the way things need to happen.

And I violently disagree.

It should not be "once of twice a kernel release".

It should be "once or twice a year" that you hit a flag-day issue. The 
rest of the time you should be able to do it without breakage. It's 
doable. You just HAVEN'T EVEN TRIED, and seem to be actively against even 
doing so.

		Linus
--


From: Roel Kluin <12o3l@...>
Subject: Re: Announce: Linux-next (Or Andrew's dream :-))
Date: Feb 13, 1:12 pm 2008

Linus Torvalds wrote:
> 
> On Tue, 12 Feb 2008, Greg KH wrote:
>>> That's the point.
>> Not it isn't.  To quote you a number of years ago:
>> 	"Linux is evolution, not intelligent design"
> 
> Umm. Have you read a lot of books on evolution?
> 
> It doesn't sound like you have.
> 
> The fact is, evolution often does odd (and "suboptimal") things exactly 
> because it does incremental changes that DO NOT BREAK at any point.

This is not entirely true if the pressure for changes are removed. For 
instance in mammals the bones in the ear are what used to be gills in fish.
When fish became amphibians the gills weren't needed as much and evolution
took a side path.

> The examples are legion. The mammalian eye has the retina "backwards", 
> with the blind spot appearing because the fundmanetal infrastructure (the 
> optical nerves) actually being in *front* of the light sensor and needing 
> a hole in the retina to get the information (and blood flow) to go to the 
> brain!
> 
> In other words, exactly *because* evolution requires "bisectability" (any 
> non-viable point in between is a dead end by definition) and does things 
> incrementally, it doesn't do big flips. It fixes the problems on an 
> incremental scale both when it comes to the details and when it comes to 
> both "details" (actual protein-coding genes that code directly for some 
> expression) and "infrastructure" (homeobox and non-coding genes).

In nature there is a lot of duplication: several copies of genes can exist
and different copies may have a distinct evolution. There is also a lot of
'junk' DNA that doesn't code for anything (although it may have regulating
functions). In there some copies of genes may remain that are inactivated,
as well as parts of virusses, slowly obtaining random mutations because
there is no pressure on the evolution of them. Some may eventually become
active again and have different functions.

The duplication also often ensures there is fallback when random mutations
are acquired and a protein is knocked out. Besides the two chromosomes
several proteins also can have overlapping functions. The result is more
like a balance. 
 
Evolution in nature and changes in code are different because in code junk
and bugs are constantly removed. In biology junk is allowed and may provide
a pool for future development. Linux development is intended and not
survival.
--


From: Linus Torvalds <torvalds@...>
Subject: Re: Announce: Linux-next (Or Andrew's dream :-))
Date: Feb 13, 2:09 pm 2008

On Wed, 13 Feb 2008, Roel Kluin wrote:
> 
> In nature there is a lot of duplication: several copies of genes can exist
> and different copies may have a distinct evolution.

This is true of very complex animals, but much less so when looking at 
things like bacteria (and arguably, any current sw project is closer to 
bacteria in complexity than anything mammalian).

In bacteria (and viruses), duplication of DNA/RNA is a big cost of living 
in general, and as a result there is *much* less junk DNA. So in an 
evolutionary sense, it's much closer to what the kernel should have (with 
occasional duplication of code and interfaces to allow new functionality, 
but rather aggressive pruning of the excess baggage).

In other words, all of these choices are a matter of "balance". In some 
areas, excess code is not a sufficient downside, and we keep even broken 
source code around with no actual function, "just because" (or rather, 
because the cost of carrying it around is so small that nobody cares). 

That's true in the kernel as in biology: check out not just deprecated 
code, but the drivers and other odds-and-ends that are explicitly marked 
as non-coding DNA (we just happen to call them BROKEN in our Kconfig).

			Linus
--

From: Greg KH <greg@...>
Subject: Re: Announce: Linux-next (Or Andrew's dream :-))
Date: Feb 12, 4:48 pm 2008

On Tue, Feb 12, 2008 at 11:55:45AM -0800, Linus Torvalds wrote:
> 
> 
> On Tue, 12 Feb 2008, Greg KH wrote:
> > 
> > > That's the point.
> > 
> > Not it isn't.  To quote you a number of years ago:
> > 	"Linux is evolution, not intelligent design"
> 
> Umm. Have you read a lot of books on evolution?
> 
> It doesn't sound like you have.
> 
> The fact is, evolution often does odd (and "suboptimal") things exactly 
> because it does incremental changes that DO NOT BREAK at any point.

Hm, I think we are in violent agreement here :)

> The examples are legion. The mammalian eye has the retina "backwards", 
> with the blind spot appearing because the fundmanetal infrastructure (the 
> optical nerves) actually being in *front* of the light sensor and needing 
> a hole in the retina to get the information (and blood flow) to go to the 
> brain!
> 
> In other words, exactly *because* evolution requires "bisectability" (any 
> non-viable point in between is a dead end by definition) and does things 
> incrementally, it doesn't do big flips. It fixes the problems on an 
> incremental scale both when it comes to the details and when it comes to 
> both "details" (actual protein-coding genes that code directly for some 
> expression) and "infrastructure" (homeobox and non-coding genes).
> 
> So quite frankly, you're the "intelligent designer" here. You're the one 
> who seems to claim that we need those leaps of faith and wild jumps.

No, I might have mistyped previously, but I really do not believe in
this at all.

> > > And simply avoiding cross-subsystem API changes unless there is a major 
> > > *MAJOR* reason for them is the obvious thing to do. Simply face the fact 
> > > that even in open source there are major reasons to stay with an old 
> > > interface even if it's not optimal.
> > 
> > I strongly disagree here.  We lived with that kset/ktype crap for years,
> > and I finally broke down and cleaned it up, simplifying things, removing
> > code, making the kernel smaller, leaner, and easier for others to change
> > and use in the future.  With your statement, such a change should have
> > never taken place as it what we had at the time was "not optimal", but
> > good enough to live with.
> 
> You didn't listen at all.
> 
> I said that the threshold should be high, not that it should be 
> impossible. I also said that we should strive for making it unnecessary to 
> have the painful total synchronization points.

I agree with this.  But I was reacting to your "there are major reasons
to stay with an old interface..." portion above, while I feel we should
always work to fixe those interfaces as best as possible.

> > But they do happen about once or twice a kernel release, just by virtue
> > of the way things need to happen.
> 
> And I violently disagree.
> 
> It should not be "once of twice a kernel release".
> 
> It should be "once or twice a year" that you hit a flag-day issue. The 
> rest of the time you should be able to do it without breakage. It's 
> doable. You just HAVEN'T EVEN TRIED, and seem to be actively against even 
> doing so.

No, not at all.

I have tried, and successfully done this many times in the past.  The
kobject change was one example: add a new function, migrate all users of
a direct pointer over to that function, after that work is all done and
in, change the structure and do the needed work afterward.  All is
bisectable completly, with no big "flag day" needed.

Same goes for a lot of USB changes.  Hell, we've been slowly moving
toward changing the USB urb callback for _years_ yet, not quite getting
there, but the groundwork is done, making it much simpler and easy for
it to happen.  Eventually things will flip over and the eventual patch
for all callback functions will be trivial and easily reviewed and
merged.

In short, I agree that flag days are bad and should be avoided at all
costs, and that slow incremental, bisectable changes are how we evolve
properly.  And that we need to always be able to do this kind of work,
and never be stuck with the "can't change an old, internal api" model.

To bring it around to the original topic, -next should help us in
finding these issues out, exactly like -mm is.

If we want to make a new rule of "only merging api or cross-subsystem
changes after everything else", that's wonderful with me, I have no
objection to it at all.

thanks,

greg k-h
--


From: Theodore Tso <tytso@...>
Subject: Re: Announce: Linux-next (Or Andrew's dream :-))
Date: Feb 12, 7:49 pm 2008

On Tue, Feb 12, 2008 at 12:48:13PM -0800, Greg KH wrote:
> On Tue, Feb 12, 2008 at 11:55:45AM -0800, Linus Torvalds wrote:
> > > 
> > > Not it isn't.  To quote you a number of years ago:
> > > 	"Linux is evolution, not intelligent design"

I think this statement has been used unfortunately as a hard and fast
rule (which we all know how much Linus loves :-) to mean, in its most
extreme form, that we should *never* try to do some careful reflection
about careful API design, and that the extremes of "no interface
without an in-tree user" applies to (a) parameters in a function call
(heck, we can always sweep through all the in-tree users to add that
extra parameter later, and thats a *good* thing because it breaks
those evil out-of-tree drivers) and (b) to not even thinking if some
particular interface (that is not needed now but which reasonably will
be needed later) is even *possible* without doing a sweep of all of
the in-tree users of the interface.

Related to this syndrome is the assumption that measuring the rate of
changes in lines of code changed per second implies that any
development process which causes the number of lines of code changed
second, including frequent sweeps through the tree changing all
interfaces, is a *good* thing.

Yes, this is an extreme position, and I'm not accusing anyone of
holding the above in its entirety --- but I've seen aspects of all of
these from one developer or another.

We come to it from the attacking another strawman, which assumes that
*all* interfaces which don't have an in-tree are evil, and that
keeping old __deprecated interfaces for a long time is an evil which
causes intolerable pain, and that it's never worthwhile to try to
anticipate future expandibility into an interface because you will
inevitably get it wrong.

Clearly, we are right to mock Solaris for making claims that they will
never, ever, *ever* change an interface, not even one that goes back
sixteen years to Solaris 2.3.  But it doesn't follow the opposite
point of view, that constant mutability of kernel interfaces to make
sure that things are always perfect and pure and clean is the right
one either.

> > The examples are legion. The mammalian eye has the retina "backwards", 
> > with the blind spot appearing because the fundmanetal infrastructure (the 
> > optical nerves) actually being in *front* of the light sensor and needing 
> > a hole in the retina to get the information (and blood flow) to go to the 
> > brain!

Also, evolution also means that things like vestigal organs (like our
appendix) are tolerated.  So are things like clearly very badly
designed things, like human backs.  To the extent that we don't like
vestigal old __deprecated interfaces, and want things to be perfect,
we are actually straying into the realms where we want the sort of
things that you would get if you *did* have an intelligent designer
designing the human body from scratch.

So the "Linux is evolution, not intelligent design" quote is
unfortunately getting used to imply that no amount of intelligent
foresight is worthwhile, and I think that's unfortunate.  It implies
an extreme position which is not warranted.

> > > But they do happen about once or twice a kernel release, just by virtue
> > > of the way things need to happen.
> > 
> > And I violently disagree.
> > 
> > It should not be "once of twice a kernel release".
> > 
> > It should be "once or twice a year" that you hit a flag-day issue. The 
> > rest of the time you should be able to do it without breakage. It's 
> > doable. You just HAVEN'T EVEN TRIED, and seem to be actively against even 
> > doing so.
> 
> No, not at all.
> 
> I have tried, and successfully done this many times in the past.  The
> kobject change was one example: add a new function, migrate all users of
> a direct pointer over to that function, after that work is all done and
> in, change the structure and do the needed work afterward.  All is
> bisectable completly, with no big "flag day" needed.

Collectively, we need to try harder.

We can debate exactly where the right line is, in terms of whether
it's only "once or twice a kernel release", or "once or twice a year",
but clearly the current amount of interface changes and cross-tree
dependencies has been causing Andrew pain.  And to me, that means we
need to turn the knob back a quarter turn towards tolerating
__deprecated old interfaces a little bit more, and trying to get
interfaces right just a little bit more and try building in just a
little bit more future expandability, and to try just *little* bit
harder to preserve a *little* bit more stable API.

In other words, maybe we need to write a counterpoint to the
stable_api_nonsense.txt and call it unstable_api_nonsense.txt --- and
in it, we note that if we start burning out Andrew and he starts
getting really, REALLY grumpy --- and if especially we start making
Stephen (normally a very mild-mannered and not terribly excitable guy)
grumpy, that it's time that we try just a little bit harder to make
our API's a little bit more stable.  Suckers^H^H^H^H^H^H^H^H, err.,
dedicated release managers like like Andrew and Stephen are very
precious resources and we shouldn't be burning them out by assuming
that "stable_api_nonsense.txt" is a license to constantly churn our
internal API's, to the point where they start complaining.

					- Ted
--