pack operation is thrashing my server

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: <git@...>
Date: Sunday, August 10, 2008 - 3:47 pm

Hi,

I'm having memory issues when trying to clone a remote git repository.

I'm running: "git clone git+ssh://user@foo.bar.com/var/git/foo"

The remote repository is bare, and is 180MB in size (says du), with
1824 objects. The remote (VPS) server is running git version 1.5.6.4
on Arch Linux on a x86_64 Opteron with 256MB of dedicated RAM.

The clone command fires off some packing operations that bring the
server to its knees:

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
21782 kenpratt  20   0  444m 212m  272 D    3 83.0   0:04.98 git-pack-object

The clone also seems to hang forever. Progress stays at 0% for hours,
and it never progresses past compressing the first object.

I've tried very conservative pack settings:

[pack]
        threads = 1
        windowmemory = 64M
        deltacachesize = 1M
        deltacachelimit = 1M

[pack]
        threads = 1
        windowmemory = 16M
        deltacachesize = 16M
        deltacachelimit = 0

I've tries many variations like those, but nothing seems to help.

A "git repack -a -d" only takes 5 seconds to run on the same
repository on my laptop (a non-bare copy), and seems to peak at ~160MB
of RAM usage.

Any tips/help would be greatly appreciated. This repository is still
small -- it will eventually grow to multiple GB in size, as it is a
mix of small text files and binaries ranging in size from 2MB to
200MB. Is it not feasible to clone repositories of that size that are
hosted on a server with 256MB of RAM?

Thanks!

Ken
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
pack operation is thrashing my server, Ken Pratt, (Sun Aug 10, 3:47 pm)
Re: pack operation is thrashing my server, Jakub Narebski, (Wed Aug 13, 8:43 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Sun Aug 10, 11:04 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:43 am)
Re: pack operation is thrashing my server, Andi Kleen, (Mon Aug 11, 3:10 pm)
Re: pack operation is thrashing my server, Geert Bosch, (Tue Aug 12, 11:12 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 10:35 am)
Re: pack operation is thrashing my server, Geert Bosch, (Wed Aug 13, 12:01 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 1:26 pm)
Re: pack operation is thrashing my server, Dana How, (Wed Aug 13, 1:13 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 10:59 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 11:43 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 11:50 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 1:04 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 1:21 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 2:38 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 2:55 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 1:58 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 3:04 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 3:44 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 5:50 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 7:14 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Fri Aug 15, 8:34 pm)
Re: pack operation is thrashing my server, Junio C Hamano, (Sat Sep 6, 9:03 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Sat Sep 6, 9:46 pm)
Re: pack operation is thrashing my server, Mike Hommey, (Sun Sep 7, 3:45 am)
Re: pack operation is thrashing my server, Jon Smirl, (Sat Sep 6, 10:50 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Sat Sep 6, 11:07 pm)
Re: pack operation is thrashing my server, Andreas Ericsson, (Sun Sep 7, 4:18 am)
Re: pack operation is thrashing my server, Jon Smirl, (Sat Sep 6, 11:43 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Sun Sep 7, 12:50 am)
Re: pack operation is thrashing my server, Jon Smirl, (Sun Sep 7, 9:58 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Sun Sep 7, 1:08 pm)
Re: pack operation is thrashing my server, Jon Smirl, (Sun Sep 7, 4:33 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Mon Sep 8, 10:17 am)
Re: pack operation is thrashing my server, Jon Smirl, (Mon Sep 8, 11:12 am)
Re: pack operation is thrashing my server, Jon Smirl, (Mon Sep 8, 12:01 pm)
Re: pack operation is thrashing my server, Junio C Hamano, (Sat Sep 6, 10:33 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Sun Sep 7, 1:11 pm)
Re: pack operation is thrashing my server, Junio C Hamano, (Sun Sep 7, 1:41 pm)
Re: pack operation is thrashing my server, Björn, (Thu Aug 14, 7:39 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 8:06 pm)
Re: pack operation is thrashing my server, Björn, (Sat Aug 16, 8:47 am)
Re: pack operation is thrashing my server, Linus Torvalds, (Thu Aug 14, 8:25 pm)
Re: pack operation is thrashing my server, Andi Kleen, (Thu Aug 14, 5:30 pm)
Re: pack operation is thrashing my server, Linus Torvalds, (Fri Aug 15, 12:15 pm)
Re: pack operation is thrashing my server, Andreas Ericsson, (Thu Aug 14, 2:33 am)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 10:01 am)
Re: pack operation is thrashing my server, Thomas Rast, (Thu Aug 14, 6:04 am)
Re: pack operation is thrashing my server, Andreas Ericsson, (Thu Aug 14, 6:15 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Thu Aug 14, 6:33 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Thu Aug 14, 9:46 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 1:19 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Tue Aug 12, 11:15 pm)
Re: pack operation is thrashing my server, Geert Bosch, (Tue Aug 12, 11:58 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 10:37 am)
Re: pack operation is thrashing my server, Jakub Narebski, (Wed Aug 13, 10:56 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Wed Aug 13, 11:04 am)
Re: pack operation is thrashing my server, Johan Herland, (Wed Aug 13, 12:10 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Wed Aug 13, 1:38 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Wed Aug 13, 1:57 pm)
Re: pack operation is thrashing my server, David Tweed, (Wed Aug 13, 11:26 am)
Re: pack operation is thrashing my server, Martin Langhoff, (Wed Aug 13, 7:54 pm)
Re: pack operation is thrashing my server, David Tweed, (Thu Aug 14, 5:04 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 3:22 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:29 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 3:34 pm)
Re: pack operation is thrashing my server, Andi Kleen, (Mon Aug 11, 4:10 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:15 pm)
Re: pack operation is thrashing my server, Nicolas Pitre, (Tue Aug 12, 10:38 pm)
Re: pack operation is thrashing my server, Andi Kleen, (Tue Aug 12, 10:50 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Tue Aug 12, 10:57 pm)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 11:01 am)
Re: pack operation is thrashing my server, Ken Pratt, (Mon Aug 11, 3:13 pm)
Re: pack operation is thrashing my server, Avery Pennarun, (Mon Aug 11, 11:40 am)
Re: pack operation is thrashing my server, Shawn O. Pearce, (Mon Aug 11, 11:59 am)
Re: pack operation is thrashing my server, Martin Langhoff, (Sun Aug 10, 7:06 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Sun Aug 10, 7:12 pm)
Re: pack operation is thrashing my server, Martin Langhoff, (Sun Aug 10, 7:30 pm)
Re: pack operation is thrashing my server, Ken Pratt, (Sun Aug 10, 7:34 pm)