[PATCH 18/23] Subject: SCHED - Optimize cpu search with hamming weight

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: <mingo@...>
Cc: <rostedt@...>, <ghaskins@...>, <linux-rt-users@...>, <linux-kernel@...>
Date: Tuesday, December 4, 2007 - 4:45 pm

We can cheaply track the number of bits set in the cpumask for the lowest
priority CPUs.  Therefore, compute the mask's weight and use it to skip
the optimal domain search logic when there is only one CPU available.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
---

 kernel/sched_rt.c |   25 ++++++++++++++++++-------
 1 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/kernel/sched_rt.c b/kernel/sched_rt.c
index fe0b43f..0514b27 100644
--- a/kernel/sched_rt.c
+++ b/kernel/sched_rt.c
@@ -301,7 +301,7 @@ static int find_lowest_cpus(struct task_struct *task, cpumask_t *lowest_mask)
 	int       cpu;
 	cpumask_t *valid_mask = &__get_cpu_var(valid_cpu_mask);
 	int       lowest_prio = -1;
-	int       ret         = 0;
+	int       count       = 0;
 
 	cpus_clear(*lowest_mask);
 	cpus_and(*valid_mask, cpu_online_map, task->cpus_allowed);
@@ -314,7 +314,7 @@ static int find_lowest_cpus(struct task_struct *task, cpumask_t *lowest_mask)
 
 		/* We look for lowest RT prio or non-rt CPU */
 		if (rq->rt.highest_prio >= MAX_RT_PRIO) {
-			if (ret)
+			if (count)
 				cpus_clear(*lowest_mask);
 			cpu_set(rq->cpu, *lowest_mask);
 			return 1;
@@ -326,14 +326,17 @@ static int find_lowest_cpus(struct task_struct *task, cpumask_t *lowest_mask)
 			if (rq->rt.highest_prio > lowest_prio) {
 				/* new low - clear old data */
 				lowest_prio = rq->rt.highest_prio;
-				cpus_clear(*lowest_mask);
+				if (count) {
+					cpus_clear(*lowest_mask);
+					count = 0;
+				}
 			}
 			cpu_set(rq->cpu, *lowest_mask);
-			ret = 1;
+			count++;
 		}
 	}
 
-	return ret;
+	return count;
 }
 
 static inline int pick_optimal_cpu(int this_cpu, cpumask_t *mask)
@@ -357,9 +360,17 @@ static int find_lowest_rq(struct task_struct *task)
 	cpumask_t *lowest_mask = &__get_cpu_var(local_cpu_mask);
 	int this_cpu = smp_processor_id();
 	int cpu      = task_cpu(task);
+	int count    = find_lowest_cpus(task, lowest_mask);
 
-	if (!find_lowest_cpus(task, lowest_mask))
-		return -1;
+	if (!count)
+		return -1; /* No targets found */
+
+	/*
+	 * There is no sense in performing an optimal search if only one
+	 * target is found.
+	 */
+	if (count == 1)
+		return first_cpu(*lowest_mask);
 
 	/*
 	 * At this point we have built a mask of cpus representing the

--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH 00/23] RT balance v7, Gregory Haskins, (Tue Dec 4, 4:44 pm)
Re: [PATCH 00/23] RT balance v7, Ingo Molnar, (Tue Dec 4, 5:27 pm)
[PATCH 0/3] RT balance v7a, Gregory Haskins, (Tue Dec 4, 10:55 pm)
[PATCH 1/3] Subject: SCHED - Add sched-domain roots, Gregory Haskins, (Tue Dec 4, 10:55 pm)
Re: [PATCH 00/23] RT balance v7, Gregory Haskins, (Tue Dec 4, 5:35 pm)
[PATCH 21/23] Subject: SCHED - Add sched-domain roots, Gregory Haskins, (Tue Dec 4, 4:46 pm)
[PATCH 16/23] Subject: SCHED - Avoid overload, Gregory Haskins, (Tue Dec 4, 4:45 pm)
[PATCH 19/23] Subject: SCHED - Optimize out cpu_clears, Gregory Haskins, (Tue Dec 4, 4:46 pm)
[PATCH 18/23] Subject: SCHED - Optimize cpu search with hamm..., Gregory Haskins, (Tue Dec 4, 4:45 pm)
[PATCH 15/23] Subject: SCHED - Optimize rebalancing, Gregory Haskins, (Tue Dec 4, 4:45 pm)
[PATCH 03/23] Subject: SCHED - push RT tasks, Gregory Haskins, (Tue Dec 4, 4:44 pm)
[PATCH 06/23] Subject: SCHED - wake up balance RT, Gregory Haskins, (Tue Dec 4, 4:44 pm)
[PATCH 05/23] Subject: SCHED - pull RT tasks, Gregory Haskins, (Tue Dec 4, 4:44 pm)