Re: CFS: some bad numbers with Java/database threading

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Antoine Martin <antoine@...>
Cc: Ingo Molnar <mingo@...>, Linux Kernel Development <linux-kernel@...>, Peter Zijlstra <a.p.zijlstra@...>, Nick Piggin <nickpiggin@...>, David Schwartz <davids@...>
Date: Friday, September 14, 2007 - 12:01 pm

[ Argh, just noticed this thread got broke and had been living a parallel
  life due to Subject: changes, dropped Cc:'s, and munged In-Reply-To:'s.
  Adding back all interested folk here. ]



Ok, it appears Antoine tested with the patch and got:

http://devloop.org.uk/documentation/database-performance/Linux-Kernels/Kernels-ManyThr...
http://devloop.org.uk/documentation/database-performance/Linux-Kernels/Kernels-ManyThr...

which leads me to believe this probably wasn't a yield problem after all, though
it would still be useful if someone with more knowledge of Java could give that
code a look over ...

Curiously, the -rc3 oddity is still plainly visible there -- how do we
explain that?
Ingo, does that oddity (and the commits that went in around -rc3 time) give some
clue as to the behaviour / characteristics of Antoine's workloads?



Ok, it's reproducible, making our job easier. Which also means, Antoine,
please do try the following git-bisecting:



I don't have access to any real/meaningful SMP systems, so I wonder how much
sense it makes in practical terms for me to try and run these tests
locally on my
little boxen ... would be helpful if someone with 4/8 CPU systems could give
Antoine's testsuite a whirl :-)



Umm, you mention _changing_ this value earlier, but it still remains the same
for every thread during every loop for a given test run -- what I'm
suggesting is
making that code do something like: Thread.sleep(random(x, y)); where
x=2, y=20 and random(x, y) returns a random integer between x and y, so all
threads sleep for different durations in every loop, but still average
out to about
~10 ms over a period. Try to vary x and y (to vary the average) and post the
resulting graphs too? CONFIG_HZ (actually, full .config) and dmesg might be
useful for us as well.

Also, like David mentioned, counting the _number_ of times the test threads
managed to execute those SQL queries is probably a better benchmark than
measuring the time it takes for threads to finish execution itself -- uniformity
in that number across threads would bring out how "fair" CFS is compared to
previous kernels, for one ...

And finally I need a clarification from you: from the code that I
read, it appears
you have *different* threads for purely inserting/selecting/updating/deleting
records from the table, right? So I think David was barking up the wrong tree
in that other reply there, where he said the *same* thread needs to execute
multiple queries on the same data, and therefore your test code is susceptible
to cache hotness and thread execution order effects ... but I don't see any
such pathologies. I could be wrong, of course ...

Which brings us to another issue -- how well does the testsuite capture real
world workloads? Wouldn't multiple threads in the real world that arbitrarily
execute insert/update/select/delete queries on the same table also need to
implement some form of locking? How would that affect the numbers?


-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
CFS: some bad numbers with Java/database threading, Antoine Martin, (Wed Sep 12, 7:10 pm)
Re: CFS: some bad numbers with Java/database threading, Ingo Molnar, (Thu Sep 13, 7:24 am)
Re: CFS: some bad numbers with Java/database threading, Ingo Molnar, (Fri Sep 14, 4:32 am)
Re: CFS: some bad numbers with Java/database threading, Satyam Sharma, (Fri Sep 14, 6:06 am)
Re: CFS: some bad numbers with Java/database threading, Satyam Sharma, (Fri Sep 14, 12:01 pm)
Re: CFS: some bad numbers with Java/database threading, Antoine Martin, (Mon Sep 17, 8:17 am)
Re: CFS: some bad numbers with Java/database threading, Satyam Sharma, (Fri Sep 14, 12:08 pm)
Re: CFS: new java yield graphs, Antoine Martin, (Tue Sep 25, 9:46 pm)
Re: CFS: new java yield graphs, Ingo Molnar, (Thu Sep 27, 4:35 am)
RE: some bad numbers with Java/database threading, David Schwartz, (Thu Sep 13, 3:18 am)
Re: some bad numbers with Java/database threading, Nick Piggin, (Wed Sep 12, 7:33 pm)
Re: some bad numbers with Java/database threading, Antoine Martin, (Thu Sep 13, 3:02 pm)
RE: some bad numbers with Java/database threading, David Schwartz, (Thu Sep 13, 5:47 pm)