Re: linux+glibc memory allocator, poor performance

Previous thread: [PATCH] x86: merge the simple bitops and move them to bitops.h by Alexander van Heukelum on Wednesday, March 12, 2008 - 1:01 pm. (20 messages)

Next thread: [PATCH] x86: parocessor.h - use PAGE_SIZE instead of numeric constant by Cyrill Gorcunov on Wednesday, March 12, 2008 - 1:14 pm. (3 messages)
From: J.C. Pizarro
Date: Wednesday, March 12, 2008 - 1:09 pm

Assume a SMP system that has 8 CPUs. The main problem of requesting
pages is the BKL (Big Kernel Lock) in this SMP system used for mutual
exclusion of the shared resource (the memory).

To solve this major problem, i propose you freely to allocate 8 local caches
of (e.g.) 2 MiB each CPU (total 2MiB x 8 CPUs = 16 MiB) acting as
8 producer buffers for globally many consumer tasks (e.g. >= 20).

When the some producer buffer is empty then it does unfrequently BKL to
allocate another 2 MiB more from the shared resource (the memory).

In the reverse, it's simple, return back the unused pages to the local buffer
of the producer, and when this full then to do BKL too to unallocate its half
to the shared resource (the memory).

   Sincerely, J.C.Pizarro
--

From: Al Viro
Date: Wednesday, March 12, 2008 - 2:12 pm

Excellent advice.  One question: why the bleeding hell would we use BKL
in allocator, frequently or not, when we do not share the crucial resource
needed to come up with such ideas?  Or are you proposing to share your

I wonder...  Are you seriously implying that nobody here had ever been able
to come up with anything better than _that_ (e.g. with an amazingly advanced
idea of googling for papers on allocators for all of two minutes)?  And that
you are so sure of it that you couldn't be arsed to check what the hell is
allocator really doing, on the off-chance that somebody might have been able
to match your brilliant intellectual feat?

And then we are told that linux-kernel regulars are mean and insulting...

FWIW, the quality of proposed "solution" is not an issue; hell, even "what
if that code is really so dumb that <....> would be an improvement" is not
a problem - the estimate is too low, but far be it from me to suggest that
one should apriori assume that unreviewed code is good.  What _really_
makes the whole thing incredibly insulting is that you had not bothered
to even look at the damn thing and just went on with "of course none of
you could have ever come up with anything better - I don't even need to
look at it to tell you that".
--

From: David Newall
Date: Wednesday, March 12, 2008 - 10:57 pm

You really are being rude and insulting.  Unnecessarily so.  Maybe
you're just recalcitrant (in the sense that President Suharto inferred,
not the sense that Prime Minister Keating meant.)
--

From: Mike Galbraith
Date: Wednesday, March 12, 2008 - 11:12 pm

FWIW, I thought it was a well deserved "Al" thump.  (not that my opinion
or sense of humor matters, but it made me chuckle mightily)

	-Mike

--

From: David Newall
Date: Wednesday, March 12, 2008 - 11:22 pm

Hostility such as that gives this list a bad reputation and drives
people away.  One can (and should) tell people that an idea is
completely wrong without being rude.

Perhaps I'm just incredibly sensitive today.
--

From: Christoph Hellwig
Date: Thursday, March 13, 2008 - 12:38 am

Or just dumb.  If you're read past post by Mr Bizzaro you'd realize that
driving him away would be a good thing.

--

From: David Newall
Date: Thursday, March 13, 2008 - 4:02 am

This is the sort of comment that gives Linux kernel developers a bad
name.  I don't plan to read his past posts, just like I don't plan to
read yours.  I don't need to read every stupid thing you've ever said to
know that I don't want anyone driven away.  Can't you just ignore him?
--

From: Pekka Enberg
Date: Thursday, March 13, 2008 - 2:10 am

Hey, why don't we create a linux-sensitive@vger.kernel.org and you
guys can move your nice and friendly chats over there...?
--

From: David Newall
Date: Thursday, March 13, 2008 - 4:04 am

Better create a linux-arseholes@vger.kernel.org, and move the attitude
there.
--

From: Alan Cox
Date: Thursday, March 13, 2008 - 5:10 am

On Thu, 13 Mar 2008 11:10:55 +0200

I think it would be far better to create linux-flame and send the hostile
stuff away. In a lot of cultures hostile responses or unfriendly
responses are taken badly. Politeness is a good thing this list needs
more of.
--

From: Pekka Enberg
Date: Thursday, March 13, 2008 - 5:48 am

Hi Alan,


Do you honestly think we should be polite to people trolling on the
LKML? The problem with kill-filing people like J.C. Pizzarro (a known
troll on the GCC mailing list at least) is that other people will
engange in discussing these "proposals" and contribute in generating
noise anyway.
--

From: J.C. Pizarro
Date: Thursday, March 13, 2008 - 6:55 am

I'm not a troll on the GCC mailing list. Some GCC's people is calling me troll.

   J.C.Pizarro
--

From: David Newall
Date: Thursday, March 13, 2008 - 12:40 pm

I do.  It's obvious.  Rudeness detracts (and reflects poorly on one's
parents.)  Good manners cost nothing.  I might also point out that it
was bad manners, not any alleged trolling, which caused this sad
discussion.  Don't be rude; rather say nothing.
--

From: Pekka Enberg
Date: Thursday, March 13, 2008 - 1:05 pm

Hi David,


Thanks for the free lesson in manners! It's much appreciated.

                        Pekka
--

From: Al Viro
Date: Thursday, March 13, 2008 - 3:50 pm

"Deliberate insult" would describe the original posting more accurately.
Whether you prefer to look at it as a show of bad manners or as trolling

Request duly noted and denied.  For the record: condescending twits of that
variety ("I can't be arsed to look at code I'm commenting on; here's my
valuable insight that is obviously beyond the mental capacity of anybody
here") can expect a harsh reply, whoever they are.  I'm yet to see l-k
regulars treating newbies that way, BTW, despite the much decried lack of
politeness.

Since you are postulating social norms, here's one you seem to have overlooked:
one who comments on code or design, be it review, suggestions of changes/
improvements, etc., ought to read the thing he is commenting on.  Whoever
it is and wherever the code in question might be.  And yes, it applies to
kernel hackers commenting on newbies' patches to the same extent as to
completely unknown folks commenting on the kernel, etc.

Breaking that rule and being caught at that is one hell of a way to earn
yourself a lousy reputation.  Combine that with arrogance and... well,
there's a term for what you become: target practice.
--

From: Stephen Clark
Date: Friday, March 14, 2008 - 5:11 am

Someone once said:
"If you don't have anything good to say, don't day anything at all".

I have been reading LKML since the mid 90's and continually see people 
jump on
what are maybe dumb/stupid questions or trolls instead of simply 
ignoring them.

My $.02
Steve

-- 

"They that give up essential liberty to obtain temporary safety, 
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty 
decreases."  (Thomas Jefferson)



--

From: Willy Tarreau
Date: Thursday, March 13, 2008 - 11:05 pm

Being polite, understanding and grateful generally encourages people
to contribute and progress. Being sometimes rude helps them understand
they went too far, and gives more credit to your politeness in normal
time because you're seen as balanced. Being always rude or always
excessively polite just assigns you a style by which people don't know
what you really think. So using all the capabilities of the language
is finally the best solution to precisely express what you think.

Willy

--

From: David Newall
Date: Thursday, March 13, 2008 - 3:50 am

*blink* D'oh! Mahatir, of course.
--

From: J.C. Pizarro
Date: Thursday, March 13, 2008 - 5:23 am

To see "Lock ordering:" in mm/filemap.c

I tell you an example of this used big kernel lock
write_lock_irq/write_unlock_irq:

     void __remove_from_page_cache(struct page *page):
      * Remove a page from the page cache and free it. Caller has to make
      * sure the page is locked and that nobody else uses it - or that usage
      * is safe.  The caller must hold a write_lock on the mapping's tree_lock.

     void remove_from_page_cache(struct page *page) {
	     struct address_space *mapping = page->mapping;

	     BUG_ON(!PageLocked(page));

	     write_lock_irq(&mapping->tree_lock);
	     __remove_from_page_cache(page);
	     write_unlock_irq(&mapping->tree_lock);
     }

     * Shared mappings 30.11.1994, 15.8.1995  Bruno.
     * 'unifying' the page and buffer cache and SMP-threaded the
        page-cache, 21.05.1999, Ingo Molnar.
     * SMP-threaded pagemap-LRU 1999, Andrea Arcangeli.

Here doesn't say multiple SMP buffers of pages for SMP.

Don't hate to me, you've not reasons to insult me or to somebody.

You're contradictory. It uses BKL or slower spinlocks/rwlocks in allocator.
It shares the crucial resource.

The physical RAM of the SMP system is a shared resource for 2 or more CPUs
(even in NUMA systems), so to avoid the race conditions, it needs to be
temporaly locked to comply the mutual exclusion of this shared resource.

I'm saying that it will try to reduce the frequency of BKLs to maximize the
OS performance in the mind of Huffman (freq.) & Amdhal (hot code).

I don't understand you saying "to share your crack pipe", what means it?

Don't point me! Don't menace me!

I know how to work internally a monolithic O.S., so i'm not googling papers from
others and i'm thinking from my brain how to work a little better some things

You're a distractor man that distracts to the people.

   J.C.Pizarro
--

From: Rik van Riel
Date: Wednesday, March 12, 2008 - 2:22 pm

On Wed, 12 Mar 2008 21:09:04 +0100

You really should read the source code before proposing ideas.

The kernel has done roughly what you describe since a little before
2.6.0.

-- 
All rights reversed.
--

From: Al Viro
Date: Wednesday, March 12, 2008 - 2:32 pm

ITYM a little before 2.4.0, and even prior to that it hadn't been under BKL.
--

Previous thread: [PATCH] x86: merge the simple bitops and move them to bitops.h by Alexander van Heukelum on Wednesday, March 12, 2008 - 1:01 pm. (20 messages)

Next thread: [PATCH] x86: parocessor.h - use PAGE_SIZE instead of numeric constant by Cyrill Gorcunov on Wednesday, March 12, 2008 - 1:14 pm. (3 messages)