login
Header Space

 
 

Re: pack operation is thrashing my server

Score:
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Andi Kleen <andi@...>
Cc: Ken Pratt <ken@...>, Shawn O. Pearce <spearce@...>, <git@...>
Date: Tuesday, August 12, 2008 - 11:12 pm

On Aug 11, 2008, at 15:10, Andi Kleen wrote:


I've always felt that keeping largish objects (say anything >1MB)
loose makes perfect sense. These objects are accessed infrequently,
often binary or otherwise poor candidates for the delta algorithm.

Many repositories are mostly well-behaved with large number of text
files that aren't overly large and compress/diff well. However, often
a few huge files creep in. These might be a 30 MB Word or PDF documents
(with lots of images of course), a bunch of artwork, some random .tgz  
files
with required tools or otherwise.

Regardless of their origin, the presence of such files in real-world  
SCMs
is a given and can ruin performance, even if they're hardly ever  
accessed
or updated. If we would leave such oddball objects loose, the pack would
be much smaller, easier to generate, faster to use and there should be  
no
memory usage issues.

   -Geert
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
pack operation is thrashing my server, Ken Pratt, (Sun Aug 10, 3:47 pm)
Re: pack operation is thrashing my server, Jakub Narebski, (Wed Aug 13, 8:43 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Sun Aug 10, 11:04 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:43 am)
Re: pack operation is thrashing my server, Andi Kleen, (Mon Aug 11, 3:10 pm)
Re: pack operation is thrashing my server, Geert Bosch, (Tue Aug 12, 11:12 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 10:35 am)
Re: pack operation is thrashing my server, Geert Bosch, (Wed Aug 13, 12:01 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 1:26 pm)
Re: pack operation is thrashing my server, Dana How, (Wed Aug 13, 1:13 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 10:59 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 11:43 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 11:50 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 1:04 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 1:21 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 2:38 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 2:55 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 1:58 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 3:04 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 3:44 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 5:50 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 7:14 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Fri Aug 15, 8:34 pm)
Re: pack operation is thrashing my server, Junio C Hamano, (Sat Sep 6, 9:03 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Sat Sep 6, 9:46 pm)
Re: pack operation is thrashing my server, Mike Hommey, (Sun Sep 7, 3:45 am)
Re: pack operation is thrashing my server, Jon Smirl, (Sat Sep 6, 10:50 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Sat Sep 6, 11:07 pm)
Re: pack operation is thrashing my server, Andreas Ericsson, (Sun Sep 7, 4:18 am)
Re: pack operation is thrashing my server, Jon Smirl, (Sat Sep 6, 11:43 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Sun Sep 7, 12:50 am)
Re: pack operation is thrashing my server, Jon Smirl, (Sun Sep 7, 9:58 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Sun Sep 7, 1:08 pm)
Re: pack operation is thrashing my server, Jon Smirl, (Sun Sep 7, 4:33 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Mon Sep 8, 10:17 am)
Re: pack operation is thrashing my server, Jon Smirl, (Mon Sep 8, 11:12 am)
Re: pack operation is thrashing my server, Jon Smirl, (Mon Sep 8, 12:01 pm)
Re: pack operation is thrashing my server, Junio C Hamano, (Sat Sep 6, 10:33 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Sun Sep 7, 1:11 pm)
Re: pack operation is thrashing my server, Junio C Hamano, (Sun Sep 7, 1:41 pm)
Re: pack operation is thrashing my server, Björn, (Thu Aug 14, 7:39 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 8:06 pm)
Re: pack operation is thrashing my server, Björn, (Sat Aug 16, 8:47 am)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 8:25 pm)
Re: pack operation is thrashing my server, Andi Kleen, (Thu Aug 14, 5:30 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Fri Aug 15, 12:15 pm)
Re: pack operation is thrashing my server, Andreas Ericsson, (Thu Aug 14, 2:33 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 10:01 am)
Re: pack operation is thrashing my server, Thomas Rast, (Thu Aug 14, 6:04 am)
Re: pack operation is thrashing my server, Andreas Ericsson, (Thu Aug 14, 6:15 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Thu Aug 14, 6:33 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 9:46 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 1:19 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Tue Aug 12, 11:15 pm)
Re: pack operation is thrashing my server, Geert Bosch, (Tue Aug 12, 11:58 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 10:37 am)
Re: pack operation is thrashing my server, Jakub Narebski, (Wed Aug 13, 10:56 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 11:04 am)
Re: pack operation is thrashing my server, Johan Herland, (Wed Aug 13, 12:10 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Wed Aug 13, 1:38 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 1:57 pm)
Re: pack operation is thrashing my server, David Tweed, (Wed Aug 13, 11:26 am)
Re: pack operation is thrashing my server, Martin Langhoff, (Wed Aug 13, 7:54 pm)
Re: pack operation is thrashing my server, David Tweed, (Thu Aug 14, 5:04 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 3:22 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:29 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 3:34 pm)
Re: pack operation is thrashing my server, Andi Kleen, (Mon Aug 11, 4:10 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:15 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Tue Aug 12, 10:38 pm)
Re: pack operation is thrashing my server, Andi Kleen, (Tue Aug 12, 10:50 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Tue Aug 12, 10:57 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 11:01 am)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:13 pm)
Re: pack operation is thrashing my server, Avery Pennarun, (Mon Aug 11, 11:40 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 11:59 am)
Re: pack operation is thrashing my server, Martin Langhoff, (Sun Aug 10, 7:06 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Sun Aug 10, 7:12 pm)
Re: pack operation is thrashing my server, Martin Langhoff, (Sun Aug 10, 7:30 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Sun Aug 10, 7:34 pm)
speck-geostationary