Re: [RFC] kbuild - introduce vdir to make life easier for x86_64

Previous thread: [PATCH] dcache: trivial comment fix by J. Bruce Fields on Monday, September 10, 2007 - 2:46 pm. (2 messages)

Next thread: [patch 0/3] futex_compat fixes by arnd on Monday, September 10, 2007 - 3:13 pm. (1 message)
To: Thomas Gleixner <tglx@...>, Andi Kleen <ak@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Monday, September 10, 2007 - 3:11 pm

Hi Andi & Thomas.

One of the complaints raised about the current x86_64
Makfiles are the ugliness needed to reuse code from i386.
Andi asked me if we could do something in kbuild to make
this less ugly and below are the hack I could come up with.

The trick is that in the Makefile we tell kbuild what directory
to search for source files should we fail to locate them in
current directory. This is done by an assingment to the variable
named 'vdir' - a counterpart to make's vpath but it takes only a single
directory and not a list.

In Makefile.build I added an additional rule (see last part of the patch)
that tell kbuild that it may find the sourcefile in vdir if not
present in current dir.
This allowed me to delete some cruft from the current x86_64 makefiles.

The below are the minimal clean-up - a bit more could be done.

Comments?

This is obviously only a RFC patch - I have not even bothered to update
the documentation to describe vdir...

Sam

arch/x86_64/kernel/Makefile | 11 +----------
arch/x86_64/mm/Makefile | 3 +--
arch/x86_64/pci/Makefile | 12 +-----------
scripts/Makefile.build | 4 ++++
4 files changed, 7 insertions(+), 23 deletions(-)

diff --git a/arch/x86_64/kernel/Makefile b/arch/x86_64/kernel/Makefile
index ff5d8c9..fe58ac6 100644
--- a/arch/x86_64/kernel/Makefile
+++ b/arch/x86_64/kernel/Makefile
@@ -1,7 +1,7 @@
#
# Makefile for the linux kernel.
#
-
+vdir := arch/i386/kernel
extra-y := head.o head64.o init_task.o vmlinux.lds
EXTRA_AFLAGS := -traditional
obj-y := process.o signal.o entry.o traps.o irq.o \
@@ -49,15 +49,6 @@ obj-y += pcspeaker.o
CFLAGS_vsyscall.o := $(PROFILING) -g0

therm_throt-y += ../../i386/kernel/cpu/mcheck/therm_throt.o
-bootflag-y += ../../i386/kernel/bootflag.o
-cpuid-$(subst m,y,$(CONFIG_X86_CPUID)) += ../../i386/kernel/cpuid.o
-topology-y += ../../i386/kernel/topology.o
-microcode-$(subst m,y,$(CONFIG_MICROCODE)) += ../../i386/kerne...

To: Sam Ravnborg <sam@...>
Cc: Thomas Gleixner <tglx@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Monday, September 10, 2007 - 6:40 pm

Looks good in principle. My only suggestion would be to name it something
differently than vdir. I know that's what GNU make calls it, but it's still
pretty cryptic. How about just fallback-dir ?

Also what would be nice (I don't know if it's doable in make) would
be a separate variable (e.g. other-obj-... := ) that contains the fallback
names and that is double checked against the fallback directory. That
would document it clearly what's going on. Failing that stuffing
them into a comment would be good at least.

-Andi

-

To: Andi Kleen <ak@...>
Cc: Thomas Gleixner <tglx@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Tuesday, September 11, 2007 - 3:42 pm

I did not like the vdir hack for a couple of reasons as partly outline by Andi.
So starting to wonder why the obvious did not work.

For mm/Makefile I wanted to do:

diff --git a/arch/x86_64/mm/Makefile b/arch/x86_64/mm/Makefile
index d25ac86..6f4addf 100644
--- a/arch/x86_64/mm/Makefile
+++ b/arch/x86_64/mm/Makefile
@@ -1,11 +1,10 @@
#
# Makefile for the linux x86_64-specific parts of the memory manager.
#
+i386 := ../../../arch/i386/mm

obj-y := init.o fault.o ioremap.o extable.o pageattr.o mmap.o
-obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
+obj-$(CONFIG_HUGETLB_PAGE) += $(i386)/hugetlbpage.o
obj-$(CONFIG_NUMA) += numa.o
obj-$(CONFIG_K8_NUMA) += k8topology.o
obj-$(CONFIG_ACPI_NUMA) += srat.o
-
-hugetlbpage-y = ../../i386/mm/hugetlbpage.o

And after wondering a bit I came up with following fix for kbuild:
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index fc498fe..ff03b15 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -46,7 +46,7 @@ multi-objs := $(multi-objs-y) $(multi-objs-m)

# $(subdir-obj-y) is the list of objects in $(obj-y) which do not live
# in the local directory
-subdir-obj-y := $(foreach o,$(obj-y),$(if $(filter-out $(o),$(notdir $(o))),$(o)))
+subdir-obj-y := $(filter %/built-in.o, $(obj-y))

# $(obj-dirs) is a list of directories that contain object files
obj-dirs := $(dir $(multi-objs) $(subdir-obj-y))

subdir-obj-y listed all targets in another dir than current dir
but only needed to list built-in.o targets.

So after fixig this long standing bug in kbuild I could do the attached
cleanup of current x86_64 Makefiles.

I will push the kbuild fix in next merge window but due to the ongoing
merge talk I dunno about the x86_64 patch.
Anyway the kbuild fix needs to be pushed first.

Sam

arch/x86_64/kernel/Makefile | 37 +++++++++++-------------------------
arch/x86_64/kernel/acpi/Makefile | 12 ++++-------
arch/x86_64/kernel/cpufreq/Makefile | 18 +++++------------
arch/x86_64/mm/M...

To: Andi Kleen <ak@...>
Cc: Thomas Gleixner <tglx@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Tuesday, September 11, 2007 - 2:49 am

fallback-dir is better.
vdir came from make's vpath but then very few knows about vpath anyway so that
My first attempt was along these lines but I dropped it for two reasons:
1) Mixing obj-y and other-obj-y assinment broke all assumptions on linkorder.
It would not be possible to link objexts in order a1.o, a2.o, a3.o if a1.o
and a3.o were assigned to obj-y and a2.o was assigned to other-obj-y.

2) It needed a decent amouth of unreadable gmake-foo to introduce
a second variable. And the learning curve for kbuild internals are alreay
to steep so I try to avoid additional complexity.

I'm a bit reluctant to just enable the fallback-dir functionality.
There is no point in adding this for the sake of x86 if we merge
i386 and x86_64 a few days after.

Sam
-

To: Andi Kleen <ak@...>
Cc: Sam Ravnborg <sam@...>, Thomas Gleixner <tglx@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Monday, September 10, 2007 - 6:50 pm

<nitpick>

Actually make calls it VPATH, not vdir.

-hpa
-

To: Sam Ravnborg <sam@...>
Cc: Andi Kleen <ak@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>, Ingo Molnar <mingo@...>
Date: Monday, September 10, 2007 - 3:22 pm

Sam,

while in general this is definitely a nice change, it does not really
solve the real problem of code scattered across two architectures. The
Makefile polishing is the least thing we care about.

Thanks,

tglx

-

To: Thomas Gleixner <tglx@...>
Cc: Sam Ravnborg <sam@...>, Andi Kleen <ak@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Monday, September 10, 2007 - 3:29 pm

i'd like to add it here that Makefile polishing is important - it's just
that in the context of arch/*x86* the Makefile impact of the current
cross-arch code sharing practice is one of the smaller problems and the
Makefiles get cleaned up via the arch/x86 merge anyway.

Ingo
-

To: Ingo Molnar <mingo@...>
Cc: Thomas Gleixner <tglx@...>, Andi Kleen <ak@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Monday, September 10, 2007 - 4:34 pm

Partly so. Took a look at the x86 tree.
The main Makefile are at least not merged. Neither are pci/Makefile not
boot/compressed/Makefile.
And some of the rest of the Makefiles are not pretty with the huge arch
specific sections ifdeffed out.

I have long thought about some extensions to the kbuild 'language'
along the following lines:

Additional shorthands for obj-m:
obj-m-if-m
obj-m-if-y
obj-m-ifn-

Additional shorthands for obj-y:
obj-y-if-m
obj-y-if-y
obj-y-ifn-

The ifn- versions are to test for empty options.

This may as an example result in following changes to the
acpi/Makefile in the merged tree:

Makefile | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kernel/acpi/Makefile b/arch/x86/kernel/acpi/Makefile
index ad4baa6..ec5a295 100644
--- a/arch/x86/kernel/acpi/Makefile
+++ b/arch/x86/kernel/acpi/Makefile
@@ -1,15 +1,11 @@
obj-$(CONFIG_ACPI) += boot.o

-ifeq ($(CONFIG_X86_32),y)
ifneq ($(CONFIG_PCI),)
-obj-$(CONFIG_X86_IO_APIC) += earlyquirk.o
-endif
-obj-$(CONFIG_ACPI_SLEEP) += sleep_32.o wakeup_32.o
-else
-obj-$(CONFIG_ACPI_SLEEP) += sleep_64.o wakeup_64.o
+obj-$(CONFIG_X86_32)-if-$(CONFIG_X86_IO_APIC) += sleep_32.o wakeup_32.o
endif

-ifneq ($(CONFIG_ACPI_PROCESSOR),)
-obj-y += cstate.o processor.o
-endif
+obj-$(CONFIG_X86_32)-if-$(CONFIG_ACPI_SLEEP) += sleep_32.o wakeup_32.o
+obj-$(CONFIG_X86_64)-if-$(CONFIG_ACPI_SLEEP) += sleep_64.o wakeup_64.o
+
+obj-y-if-$(CONFIG_ACPI_PROCESSOR) += cstate.o processor.o

My biggest worry is that we end up with a more compact format
but only me (and a very few others) can read it.
But I think the above could make the x86 Makefiles more readable
as a whole.

Sam
-

To: Sam Ravnborg <sam@...>
Cc: Ingo Molnar <mingo@...>, Andi Kleen <ak@...>, LKML <linux-kernel@...>, kbuild devel <kbuild-devel@...>
Date: Monday, September 10, 2007 - 4:44 pm

Sam,

Yeah I know. Those are the non trivial ones and the boot/compressed one
might be split forever. The pci Makefile has link order problems
(initcall order wreckage) and the main Makefile as well. Needs more

It's way better than the ifneq(...) stuff and completely understandable
at least for me. I'd like to see that change, it is helpful on a bunch
of other places in the kernel as well.

Thanks,

tglx

-

Previous thread: [PATCH] dcache: trivial comment fix by J. Bruce Fields on Monday, September 10, 2007 - 2:46 pm. (2 messages)

Next thread: [patch 0/3] futex_compat fixes by arnd on Monday, September 10, 2007 - 3:13 pm. (1 message)