Re: git and larger trees, not so fast?

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Junio C Hamano <gitster@...>
Cc: Sean <seanlkml@...>, moe <moe-git@...>, <git@...>
Date: Thursday, August 9, 2007 - 8:44 pm

On Thu, 9 Aug 2007, Junio C Hamano wrote:


Ugh.

I had some time, so I tried to find out *why* that thing is so slow.

The fact is, "git read-tree -m HEAD" should be really really fast, because 
it should never actually insert multiple entries into the same index 
entry: it should just _replace_ the entry.

But why is it slow?

It doesn't actually replace the entry with "add_cache_entry()" at all. 
What it does is to *remove* the entry entirely at unpack-trees.c, line 
154, unpack_trees_rec(), which does a "remove_cache_entry_at(o->pos);".

That causes us to have to condense the index array, and is one big 
memcpy() for a large index.

It then ADDS THE NEW ENTRY BACK! Which causes *another* expensive index 
array memmove(), as it now needs to make room (at the same location that 
it just compacted).

Sadly, that removal is required for some of the other cases, so it's not 
like we can remove the remove. But we could *possibly* make things 
ridiculously much faster by making the remove a lazy thing, and if the 
next index operation just adds it back in, we wouldn't move things around.

A bit too subtle for my taste.

		Linus
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
git and larger trees, not so fast?, moe, (Thu Aug 9, 12:30 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Sat Aug 11, 2:47 pm)
Re: git and larger trees, not so fast?, moe, (Sat Aug 11, 4:06 pm)
Re: git and larger trees, not so fast?, moe, (Wed Aug 22, 8:30 pm)
Re: git and larger trees, not so fast?, Fernando J. Pereda, (Sat Aug 11, 3:02 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Sat Aug 11, 4:38 pm)
Re: git and larger trees, not so fast?, Fernando J. Pereda, (Sat Aug 11, 4:51 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Sat Aug 11, 6:27 pm)
Re: git and larger trees, not so fast?, David Kastrup, (Sat Aug 11, 7:26 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Fri Aug 10, 3:39 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 1:11 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 1:54 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 1:38 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 2:06 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 2:11 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 4:42 pm)
Re: git and larger trees, not so fast?, Sean, (Thu Aug 9, 4:52 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 5:41 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 5:46 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 6:02 pm)
Re: git and larger trees, not so fast?, Daniel Barkalow, (Thu Aug 9, 9:42 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 7:38 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 8:44 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 8:51 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 8:57 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 11:48 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Fri Aug 10, 12:07 pm)
Fix "git commit directory/" performance anomaly, Linus Torvalds, (Fri Aug 10, 12:51 pm)
Re: Fix "git commit directory/" performance anomaly, Junio C Hamano, (Fri Aug 10, 2:31 pm)
Re: Fix "git commit directory/" performance anomaly, Linus Torvalds, (Fri Aug 10, 2:56 pm)
Re: Fix "git commit directory/" performance anomaly, Linus Torvalds, (Fri Aug 10, 1:14 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Fri Aug 10, 1:55 am)
Re: git and larger trees, not so fast?, Linus Torvalds, (Fri Aug 10, 11:49 am)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 8:04 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 5:37 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 2:00 pm)