Re: Linux v2.6.24-rc1, x86 arch code quality, unifications

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Christoph Hellwig <hch@...>
Cc: Linus Torvalds <torvalds@...>, Linux Kernel Mailing List <linux-kernel@...>, Thomas Gleixner <tglx@...>, H. Peter Anvin <hpa@...>
Date: Wednesday, October 24, 2007 - 6:17 am

* Christoph Hellwig <hch@infradead.org> wrote:


to answer that question one should first be aware of the fundamental 
code quality problems that the unified x86 architecture has inherited 
from the split i386 and x86_64 architectures.

To get objective and automated metrics about code quality, i've 
constructed a table of "coding style errors per one thousand lines of 
source code" numbers with the help of the latest checkpatch.pl. The 
codebases i measured are the pre-merge i386 and x86_64 tree, the 
post-merge arch/x86 unified architecture, and i've also added a handful 
of other architectures and selected core subsystems, as comparison:

               -------------------------------------------------------
               |    errors  | lines of code  |    errors/KLOC
               |            |                |    (smaller is better)
 --------------|------------|----------------|------------------------
  arch/i386/          5717            73865         77.3
  arch/x86_64/        2993            31155         96.0
  arch/x86/           8504           114654         74.1
 ..............|............|................|........................
  arch/ia64/          1779            64022         27.7
  arch/mips/          2110            94692         22.2
  arch/sparc64/       1387            49253         28.1
 ..............|............|................|........................
  kernel/              762            83540          9.1
  kernel/time/          15             4191          3.5
  kernel/irq/            1             2317          0.4
  mm/                  464            46324         10.0
  net/core             176            24413          7.2
 ..............|............|................|........................

a couple of observations. Firstly, it is plainly obvious that the x86_64 
and i386 architectures were in a dreadful state of code quality before 
the unification. Their code quality was almost an order of magnitude 
worse than that of the core kernel (!) - and their code quality was 
significantly worse than that of a couple of other, comparable 
architectures. (we knew this when we started the x86 unification effort 
- but i suspect it's even more apparent via the hard numbers in this 
table.)

 ( Note: code metrics should be taken with a grain of salt, as they
   often over-simplify the picture, but in this particular situation the
   trends are clear and the numbers match my personal impressions of
   code quality and robustness of these codebases. )

paradoxically the x86_64 architecture that had a _worse_ code quality 
than the "legacy" 32-bit code - so much about the "newer code must be 
better" misconception. The first, mechanic round of unifications thus 
brought a net degradation in quality - but we've reversed that trend in 
2.6.24-rc1 already, via unifications and cleanups, as it can be seen 
from the table. (and we did that while adding new features like 
high-resolution timers and dynticks to the x86-64bit architecture in 
v2.6.24-rc1 - or the new IOMMU code. So the x86 architecture is not 
standing still at all while the unification is going on.)

so to answer your question: full unification is no easy task and it is 
not automatic at all. The x86_64 tree has diverged from the i386 tree in 
the past 5 years due to their illogical, forced separation and a 
resulting bitrot. The two architectures have grown different sets of 
cleanliness problems and different sets of functions with arbitrary 
differences that often cover the same functionality. It's all compounded 
by the fact that the 64-bit code is in worse shape than the 32-bit - so 
it's not like we could just pick the 64-bit code and use that as the 
unified code. The 32-bit code is also used about 8-10 times more 
frequently than the 64-bit code. So there is no easy "just unify it" 
path.

The new maintainers of the x86 architecture (Thomas, Peter and me), and 
many other x86 developers are highly motivated to improve the x86 
architecture's code quality and unify the heck out of it, and there are 
some real improvements in 2.6.24-rc1 already, but we _must_ be (and are) 
working on this carefully. So we do unifications on a case by case 
basis, with the highest priority being to not introduce "unification 
regressions". The x86 architecture is the most common Linux architecture 
after all - and users care much more about having a working kernel than 
they care about cleanups and unifications. So yes, we agree with you, 
but please be patient! :-) This cannot be realistically finished in 
v2.6.24, without upsetting the codebase.

	Ingo
-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Linux v2.6.24-rc1, Linus Torvalds, (Wed Oct 24, 12:19 am)
Re: Linux v2.6.24-rc1, , (Thu Oct 25, 8:45 am)
2.6.24-rc1 doesn't build..., Theodore Tso, (Thu Oct 25, 1:18 am)
Re: 2.6.24-rc1 doesn't build..., Kamalesh Babulal, (Thu Oct 25, 1:30 am)
[patch] portman2x4.c: fix boot hang, Ingo Molnar, (Wed Oct 24, 3:44 pm)
Re: [patch] portman2x4.c: fix boot hang, Takashi Iwai, (Thu Oct 25, 4:16 am)
Re: [patch] portman2x4.c: fix boot hang, Frans Pop, (Wed Oct 24, 4:12 pm)
Re: [patch] portman2x4.c: fix boot hang, Ingo Molnar, (Wed Oct 24, 5:29 pm)
Re: Linux v2.6.24-rc1, Giacomo Catenazzi, (Wed Oct 24, 2:19 pm)
Re: [Bug 9246] On 2.6.24-rc1-gc9927c2b BUG: unable to handle..., Giacomo A. Catenazzi, (Tue Dec 4, 12:47 pm)
Re: [Bug 9246] On 2.6.24-rc1-gc9927c2b BUG: unable to handle..., Giacomo A. Catenazzi, (Wed Dec 5, 5:26 am)
2.6.24-rc1 fails with lockup and BUG:, Romano Giannetti, (Wed Oct 24, 9:25 am)
Re: 2.6.24-rc1 fails with lockup and BUG:, Joseph Fannin, (Wed Oct 24, 12:44 pm)
Re: 2.6.24-rc1 fails with lockup and BUG:, Romano Giannetti, (Fri Oct 26, 1:59 am)
Re: 2.6.24-rc1 fails with lockup and BUG:, Ingo Molnar, (Wed Oct 24, 10:27 am)
Re: 2.6.24-rc1 fails with lockup and BUG:, Romano Giannetti, (Wed Oct 24, 11:53 am)
Re: 2.6.24-rc1 fails with lockup and BUG:, Ingo Molnar, (Wed Oct 24, 11:55 am)
Re: 2.6.24-rc1 fails with lockup and BUG:, Peter Zijlstra, (Wed Oct 24, 12:11 pm)
Re: 2.6.24-rc1 fails with lockup and BUG:, Romano Giannetti, (Fri Oct 26, 1:57 am)
Re: 2.6.24-rc1 fails with lockup - /sbin/ifconfig / inet_ioc..., Stephen Hemminger, (Fri Oct 26, 12:48 pm)
[PATCH] r8169: don't call napi_disable if not doing NAPI, Stephen Hemminger, (Fri Oct 26, 2:33 pm)
Re: [PATCH] r8169: don't call napi_disable if not doing NAPI, Romano Giannetti, (Mon Oct 29, 4:56 am)
Re: [PATCH] r8169: don't call napi_disable if not doing NAPI, Romano Giannetti, (Sun Oct 28, 6:18 pm)
Re: [PATCH] r8169: don't call napi_disable if not doing NAPI, Francois Romieu, (Fri Oct 26, 4:17 pm)
[git pull] x86 arch updates, Ingo Molnar, (Wed Oct 24, 7:30 am)
Re: [git pull] x86 arch updates, Jeff Garzik, (Wed Oct 24, 7:48 am)
Re: [git pull] x86 arch updates, Ingo Molnar, (Wed Oct 24, 8:03 am)
Re: Linux v2.6.24-rc1, Christoph Hellwig, (Wed Oct 24, 4:04 am)
Re: Linux v2.6.24-rc1, Sam Ravnborg, (Wed Oct 24, 7:07 am)
Re: Linux v2.6.24-rc1, Ingo Molnar, (Wed Oct 24, 8:12 am)
Re: Linux v2.6.24-rc1, Sam Ravnborg, (Wed Oct 24, 8:21 am)
Re: Linux v2.6.24-rc1, x86 arch code quality, unifications, Ingo Molnar, (Wed Oct 24, 6:17 am)
Re: Linux v2.6.24-rc1, Ingo Molnar, (Wed Oct 24, 4:04 am)
Re: Linux v2.6.24-rc1, Dave Young, (Wed Oct 24, 1:22 am)
Re: Linux v2.6.24-rc1, Ingo Molnar, (Wed Oct 24, 3:23 am)
Re: Linux v2.6.24-rc1, Jens Axboe, (Wed Oct 24, 4:12 am)
Re: Linux v2.6.24-rc1, Dave Young, (Wed Oct 24, 3:33 am)
Re: Linux v2.6.24-rc1, Ohad Ben-Cohen, (Wed Oct 24, 3:32 am)
Re: Linux v2.6.24-rc1, Willy Tarreau, (Wed Oct 24, 12:49 am)