Re: [PATCH] RFC: git lazy clone proof-of-concept

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Jon Smirl <jonsmirl@...>
Cc: Johannes Schindelin <Johannes.Schindelin@...>, Nicolas Pitre <nico@...>, Jan Holesovsky <kendy@...>, Jakub Narebski <jnareb@...>, <git@...>, Junio C Hamano <gitster@...>
Date: Tuesday, February 12, 2008 - 6:25 pm

On Tue, 12 Feb 2008, Linus Torvalds wrote:

That's not really true, of course. But my (broken and inexact) logic is 
that we get one cost multiplier from the number of objects, and one from 
the size of the objects.

So *if* we have the situation of not limiting the window size, we 
basically have a big slowdown from raising the window in number of 
objects: not only do we get a slowdown from comparing more objects, we 
spend relatively more time comparing the *large* ones to begin with and 
having more of them just makes it even more skewed - when we hit a series 
of big blocks, the window will also contain more big blocks, so it kind of 
a double whammy.

But I don't think calling it O(windowsize^2) is really correct. It's still 
O(windowsize), it's just that the purely "number-of-object" thing doesn't 
account for big objects being much more expensive to diff. So you really 
want to make the *memory* limiter the big one, because that's the one that 
actually approximates how much time you end up spending.

So ignore that O(n^2) blather. It's not correct. What _is_ correct is that 
we want to aggressively limit memory size, because CPU cost goes up 
linearly not just with number of objects, but also super-linearly with 
size of the object ("super-linear" due to bad cache behavior and in worst 
case due to paging).

			Linus
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Fri Feb 8, 1:28 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Fri Feb 8, 4:16 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Fri Feb 8, 3:00 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 11:27 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Sun Feb 10, 9:20 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sat Feb 9, 11:10 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 12:43 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 3:50 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brandon Casey, (Thu Feb 14, 3:41 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Thu Feb 14, 4:11 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 3:58 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 2:47 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Tue Feb 12, 4:37 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 3:20 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Fri Feb 15, 5:34 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Thu Feb 14, 4:05 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brandon Casey, (Thu Feb 14, 5:08 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 5:04 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Thu Feb 14, 5:59 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Fri Feb 15, 5:43 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 7:38 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Thu Feb 14, 9:07 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brian Downing, (Thu Feb 14, 7:51 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 8:08 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Thu Feb 14, 9:41 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Shawn O. Pearce, (Sun Feb 17, 4:18 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 17, 2:44 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Junio C Hamano, (Sun Feb 17, 5:05 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brian Downing, (Thu Feb 14, 7:57 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Thu Feb 14, 4:16 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Tue Feb 12, 5:25 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 5:08 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Tue Feb 12, 5:36 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 5:59 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 6:25 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Tue Feb 12, 6:43 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 7:39 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Tue Feb 12, 5:05 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 3:42 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Sun Feb 10, 4:11 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Sun Feb 10, 1:01 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 1:36 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 1:22 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Sun Feb 10, 9:42 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 10:04 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Mon Feb 11, 6:11 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Fri Feb 8, 3:26 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Harvey Harrison, (Fri Feb 8, 4:19 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Fri Feb 8, 4:24 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Harvey Harrison, (Fri Feb 8, 4:25 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Fri Feb 8, 4:41 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Fri Feb 8, 4:09 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Andreas Ericsson, (Mon Feb 11, 6:13 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Mike Hommey, (Fri Feb 8, 2:49 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 11:06 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Fri Feb 8, 3:04 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Fri Feb 8, 2:20 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Harvey Harrison, (Fri Feb 8, 2:14 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 10:27 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Fri Feb 8, 2:03 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 10:25 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Marco Costalba, (Sun Feb 10, 3:23 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 8:08 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, David Symonds, (Sun Feb 10, 12:46 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 1:45 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 3:45 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 4:32 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Mike Hommey, (Sat Feb 9, 6:05 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sat Feb 9, 7:38 pm)