Re: git and larger trees, not so fast?

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Junio C Hamano <gitster@...>
Cc: Sean <seanlkml@...>, moe <moe-git@...>, <git@...>
Date: Friday, August 10, 2007 - 12:07 pm

On Thu, 9 Aug 2007, Junio C Hamano wrote:

You are, of course, mostly right. Using "-m" there is largely pointless, 
since it's a throw-away index, and we'll only ever use the exact paths 
that were given to us.

However, it does actually matter for one case: the case where you give a 
directory name or other name pattern, resulting in a *lot* of filenames. 
In that case, the commit will end up piping that (potentially very large) 
list to "git update-index --add --remove --stdin", and that will now mean 
that they *all* get their SHA1's recomputed.

Of course, that was the other performance bug that we already knew about 
(except we were thining "git add .", and fixed that case). So we're 
already slow at it - but we *shouldn't* be.

Try this on the kernel archive (use a clean one, so these things *should* 
all be no-ops):

	time sh -c "git add . ; git commit"

which is nice and fast and takes just over a second for me, but then try

	time git commit .

which *should* be nice and fast, but it takes forever, because we now 
re-compute all the SHA1's for *every* file. Of course, if it's all in the 
cache, it's still just 4s for me, but I tried with a cold cache, and it 
was over half a minute!

(I don't actually ever do something like "git commit .", but I could see 
people doing it. What I *do* do is that if I have multiple independent 
changes, I may actually do "git commit fs" to commit just part of them, 
and rather than list all the files, I literally just say "commit that 
sub-tree". So this really is another valid performance issue).

Sad.

			Linus
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
git and larger trees, not so fast?, moe, (Thu Aug 9, 12:30 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Sat Aug 11, 2:47 pm)
Re: git and larger trees, not so fast?, moe, (Sat Aug 11, 4:06 pm)
Re: git and larger trees, not so fast?, moe, (Wed Aug 22, 8:30 pm)
Re: git and larger trees, not so fast?, Fernando J. Pereda, (Sat Aug 11, 3:02 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Sat Aug 11, 4:38 pm)
Re: git and larger trees, not so fast?, Fernando J. Pereda, (Sat Aug 11, 4:51 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Sat Aug 11, 6:27 pm)
Re: git and larger trees, not so fast?, David Kastrup, (Sat Aug 11, 7:26 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Fri Aug 10, 3:39 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 1:11 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 1:54 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 1:38 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 2:06 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 2:11 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 4:42 pm)
Re: git and larger trees, not so fast?, Sean, (Thu Aug 9, 4:52 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 5:41 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 5:46 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 6:02 pm)
Re: git and larger trees, not so fast?, Daniel Barkalow, (Thu Aug 9, 9:42 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 7:38 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 8:44 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 8:51 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Thu Aug 9, 8:57 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 11:48 pm)
Re: git and larger trees, not so fast?, Linus Torvalds, (Fri Aug 10, 12:07 pm)
Fix "git commit directory/" performance anomaly, Linus Torvalds, (Fri Aug 10, 12:51 pm)
Re: Fix "git commit directory/" performance anomaly, Junio C Hamano, (Fri Aug 10, 2:31 pm)
Re: Fix "git commit directory/" performance anomaly, Linus Torvalds, (Fri Aug 10, 2:56 pm)
Re: Fix "git commit directory/" performance anomaly, Linus Torvalds, (Fri Aug 10, 1:14 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Fri Aug 10, 1:55 am)
Re: git and larger trees, not so fast?, Linus Torvalds, (Fri Aug 10, 11:49 am)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 8:04 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 5:37 pm)
Re: git and larger trees, not so fast?, Junio C Hamano, (Thu Aug 9, 2:00 pm)