Mark init_workqueues() as early_initcall() and thus it will be initialized
before smp bringup. init_workqueues() registers for the hotcpu notifier
and thus it should cope with the processors that are brought online after
the workqueues are initialized.

x86 smp bringup code uses workqueues and uses a workaround for the
cold boot process (as the workqueues are initialized post smp_init()).
Marking init_workqueues() as early_initcall() will pave the way for
cleaning up this code.

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
---
 include/linux/workqueue.h |    1 -
 init/main.c               |    2 --
 kernel/workqueue.c        |    4 +++-
 3 files changed, 3 insertions(+), 4 deletions(-)

Index: tip/init/main.c
===================================================================
--- tip.orig/init/main.c
+++ tip/init/main.c
@@ -32,7 +32,6 @@
 #include <linux/start_kernel.h>
 #include <linux/security.h>
 #include <linux/smp.h>
-#include <linux/workqueue.h>
 #include <linux/profile.h>
 #include <linux/rcupdate.h>
 #include <linux/moduleparam.h>
@@ -788,7 +787,6 @@ static void __init do_initcalls(void)
  */
 static void __init do_basic_setup(void)
 {
-	init_workqueues();
 	cpuset_init_smp();
 	usermodehelper_init();
 	init_tmpfs();
Index: tip/include/linux/workqueue.h
===================================================================
--- tip.orig/include/linux/workqueue.h
+++ tip/include/linux/workqueue.h
@@ -234,7 +234,6 @@ extern int schedule_on_each_cpu(work_fun
 extern int current_is_keventd(void);
 extern int keventd_up(void);
 
-extern void init_workqueues(void);
 int execute_in_process_context(work_func_t fn, struct execute_work *);
 
 extern int flush_work(struct work_struct *work);
Index: tip/kernel/workqueue.c
===================================================================
--- tip.orig/kernel/workqueue.c
+++ ...
[ message continues ]

On Fri, 30 Jul 2010 14:57:37 -0700

I sure hope this has been tested against linux-next. 
kernel/workqueue.c has been vastly changed and -tip doesn't know about
that.  linux-next should include -tip and is hence a better tree to
develop and test against.

AFAICT the main thing which needs checking is that the new
init_workqueues() doesn't do anything which requires that
sched_init_smp() has been executed.

The patch otherwise looks OK and killing that hack in the x86 code was
most merciful.

for_each_gcwq_cpu(), for_each_online_gcwq_cpu() and for_each_cwq_cpu()
make me cry.
--

[ message continues ]

hmm.. too many changes in linux-next.

Yes, as far as these patches are concerned, they work against linux-next
aswell. Just posted -v2 which is on top of linux-next. ia64 also had the
same issue, addressed in -v2 aswell.

thanks,
suresh



--

[ message continues ]

Hello,


I think it'll be best to route the changes through workqueue tree.
I'll audit init path and if everything looks okay will add the
workqueue patch to wq#for-next.

Thank you.

-- 
tejun
--

[ message continues ]

Hello,


Yeah, those are new additions for unbound workqueue support and I
should have added comments there.  Will do so.

Thanks.

-- 
tejun
--

[ message continues ]

for_each_*cwq_cpu() are similar to regular CPU iterators except that
it also considers the pseudo CPU number used for unbound workqueues.
Explain them.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
---
 kernel/workqueue.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index e5cb7fa..1105c47 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -271,6 +271,19 @@ static inline int __next_wq_cpu(int cpu, const struct cpumask *mask,
 	return __next_gcwq_cpu(cpu, mask, !(wq->flags & WQ_UNBOUND) ? 1 : 2);
 }

+/*
+ * CPU iterators
+ *
+ * An extra gcwq is defined for an invalid cpu number
+ * (WORK_CPU_UNBOUND) to host workqueues which are not bound to any
+ * specific CPU.  The following iterators are similar to
+ * for_each_*_cpu() iterators but also considers the unbound gcwq.
+ *
+ * for_each_gcwq_cpu()		: possible CPUs + WORK_CPU_UNBOUND
+ * for_each_online_gcwq_cpu()	: online CPUs + WORK_CPU_UNBOUND
+ * for_each_cwq_cpu()		: possible CPUs for bound workqueues,
+ *				  WORK_CPU_UNBOUND for unbound workqueues
+ */
 #define for_each_gcwq_cpu(cpu)						\
 	for ((cpu) = __next_gcwq_cpu(-1, cpu_possible_mask, 3);		\
 	     (cpu) < WORK_CPU_NONE;					\
-- 
1.7.1

--

[ message continues ]

From 6ee0578b4daaea01c96b172c6aacca43fd9807a6 Mon Sep 17 00:00:00 2001
From: Suresh Siddha <suresh.b.siddha@intel.com>
Date: Fri, 30 Jul 2010 14:57:37 -0700

Mark init_workqueues() as early_initcall() and thus it will be initialized
before smp bringup. init_workqueues() registers for the hotcpu notifier
and thus it should cope with the processors that are brought online after
the workqueues are initialized.

x86 smp bringup code uses workqueues and uses a workaround for the
cold boot process (as the workqueues are initialized post smp_init()).
Marking init_workqueues() as early_initcall() will pave the way for
cleaning up this code.

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
---
Audited and tested.  It should be fine.  Patch applied and pushed out
to linux-next.  How shall we route the second patch?

Thanks.

 include/linux/workqueue.h |    1 -
 init/main.c               |    2 --
 kernel/workqueue.c        |    4 +++-
 3 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h
index 5f76001..51dc9a7 100644
--- a/include/linux/workqueue.h
+++ b/include/linux/workqueue.h
@@ -327,7 +327,6 @@ extern int schedule_delayed_work_on(int cpu, struct delayed_work *work,
 extern int schedule_on_each_cpu(work_func_t func);
 extern int keventd_up(void);

-extern void init_workqueues(void);
 int execute_in_process_context(work_func_t fn, struct execute_work *);

 extern int flush_work(struct work_struct *work);
diff --git a/init/main.c b/init/main.c
index 3bdb152..5f2ec2c 100644
--- a/init/main.c
+++ b/init/main.c
@@ -32,7 +32,6 @@
 #include <linux/start_kernel.h>
 #include <linux/security.h>
 #include <linux/smp.h>
-#include <linux/workqueue.h>
 #include <linux/profile.h>
 #include <linux/rcupdate.h>
 #include <linux/moduleparam.h>
@@ -786,7 +785,6 @@ static void __init ...
[ message continues ]

Tejun, v2 version of the second patch had both x86 and ia64 changes.

http://marc.info/?l=linux-kernel&m=128053714303108&w=2

So I am ok if you carry this with the rest of your workqueue changes
that touch these same files.

thanks,
suresh

--

[ message continues ]

Andrew seemed to have already picked up the patch.  I've sent the pull
request with the first patch.

Thank you.

-- 
tejun
--

[ message continues ]