Home » Mailing list archives » git » 2006 » November » 17

Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?

Previous thread: git-blame: handling of revisions and filenames by Alex Riesen on Wednesday, November 15, 2006 - 5:52 pm. (2 messages)

Next thread: Re: Cleaning up git user-interface warts by Shawn Pearce on Wednesday, November 15, 2006 - 7:15 pm. (3 messages)
[view original message]
From: Timur Tabi <timur@...>
To: <git@...>
Subject: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Wednesday, November 15, 2006 - 6:11 pm

After doing a "make mrproper" in my Linux git tree, the result is still 1.1GB

of files.  Compare that with just the tarball, which is just one-forth the size.


Is there a way to "trim away" old commits from the repository, so that it just

doesn't take up that much space?  I don't care about any commits made in 2005.

  As long as I can still do "git pull" from the source repo to update mine,

that's good enough.


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Thomas Kolejka <Thomas.Kolejka@...>
To: Timur Tabi <timur@...>
Cc: <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Friday, November 17, 2006 - 6:36 am

-------- Original-Nachricht --------

Datum: Wed, 15 Nov 2006 16:11:57 -0600

Von: Timur Tabi <timur@freescale.com>

An: git@vger.kernel.org


Is it possible to do this with shallow clone?

Shallow clone the local repository my.git (which should be trimmed) starting from the last needed commit to a new local repository my_trimmed.git. And then remove my.git (with something like rm -rf my.git) and rename my_trimmed.git to my.git?


Thomas

--

Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!

Ideal f
[view original message]
From: Timur Tabi <timur@...>
To: Thomas Kolejka <Thomas.Kolejka@...>
Cc: <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 12:29 pm

Maybe.  How do you do a shallow clone?  I tried "git clone" followed by

"git-repack", and that helped a lot, but the result was still twice the size of

a normal tarball.  I don't see any "shallow" option to the clone command, and

git-shallow-pack doesn't exist on my installation.


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Shawn Pearce <spearce@...>
To: Timur Tabi <timur@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 12:32 pm

Twice the size of a normal tarball isn't too bad, considering that

you have the _complete_ history in the pack and yet the normal

tarball has no history at all.


Shallow clone is a development feature still being working on in

Junio's 'pu' branch of git.git.  It has a few issues still to be

worked out so it hasn't been made part of one of the more stable

branches yet (like 'next', 'master', or 'maint').


--

Shawn.

-
[view original message]
From: Timur Tabi <timur@...>
To: Shawn Pearce <spearce@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 12:52 pm

That is true.  However, for the particular project I'm working on, double the

size is not really acceptable.


My goal is to provide a source tree that is "git enabled", so that user can use

git command to fetch, apply, and create patches.  Currently, we're just handing 


Well, until it's available on an official git release, it doesn't help me.


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Johannes Schindelin <Johannes.Schindelin@...>
To: Timur Tabi <timur@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, Shawn Pearce <spearce@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 12:57 pm

Hi,


Well, if you help it, it will be available on an official git release

soon!


Ciao,

Dscho


-
[view original message]
From: Shawn Pearce <spearce@...>
To: Timur Tabi <timur@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 12:56 pm

One of the reasons its hanging out in 'pu' still is that there is

a lack of people who are interested in the feature, and thus not

enough people are testing it.  Perhaps you might be able to lend

a hand in that regard?


--

Shawn.

-
[view original message]
From: Timur Tabi <timur@...>
To: Shawn Pearce <spearce@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 1:01 pm

Sure, I can do that!  Can you give me some pointers?  I've never done

development on git itself, so I don't know Junio or his pu (sorry, I couldn't

resist :-)).


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Shawn Pearce <spearce@...>
To: Timur Tabi <timur@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 2:39 pm

Junio C Hamano is the Git maintainer.  His published Git repository

is here:


	http://www.kernel.org/pub/scm/git/git.git/


though it oddly has pack files from Jun 2006 in the wrong directory.

Weird.  Anyway...


If you clone from that URL, or better though the native Git protocol:


	git://www.kernel.org/pub/scm/git/git.git


you will get a branch called `pu`, which is the set of "Proposed

Updates" to Git that Junio and others are currently working on.

You can then checkout a branch off that, and build it:


	git checkout -b pu-build pu

	make


finally you can either run from that directory (see INSTALL file)

or you can install the binary somewhere else.  We don't really

recommend using `pu` for production level work, so make sure you

have a backup of any repository you run it on.  :)


--

Shawn.

-
[view original message]
From: Timur Tabi <timur@...>
To: Shawn Pearce <spearce@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 5:49 pm

So how do I make a shallow clone?  I've set it all up, but there is no

git-shallow-clone command, and git help clone doesn't have anything either.


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Johannes Schindelin <Johannes.Schindelin@...>
To: Timur Tabi <timur@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, Shawn Pearce <spearce@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 6:06 pm

Hi,


Try "git clone --depth 1 <url>". This will cut each ancestor chain after

one ancestor (IIRC).


Ciao,

Dscho


-
[view original message]
From: Timur Tabi <timur@...>
To: Johannes Schindelin <Johannes.Schindelin@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, Shawn Pearce <spearce@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 6:47 pm

I think you mean git-clone.sh instead of git-clone.  If I do the above command,

I get:


$ ./git clone --depth 1 git://127.0.0.1/temp/u-boot-83xx/

Usage: /home/b04825/bin/git-clone [--template=<template_directory>]

[--use-separate-remote] [--reference <reference-repo>] [--bare] [-l [-s]] [-q]

[-u <upload-pack>] [--origin <name>] [-n] <repo> [<dir>]


However, git-clone.sh is not quite working either.  I had to run git-daemon on

my machine, because git-clone.sh doesn't like the http protocol, and my firewall

blocks everything but that.  So I cloned a repo, started git-daemon, and I tried

this:


$ ./git-clone.sh --depth 1 git://127.0.0.1/temp/u-boot-83xx

usage: git-fetch-pack [--all] [-q] [-v] [-k] [--thin] [--exec=upload-pack]

[host:]directory <refs>...

fetch-pack from 'git://127.0.0.1/temp/u-boot-83xx/' failed.


A regular git-clone of git://127.0.0.1/temp/u-boot-83xx works, so I think

there's something wrong with git-clone.sh or my invocation thereof.


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Johannes Schindelin <Johannes.Schindelin@...>
To: Timur Tabi <timur@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, Shawn Pearce <spearce@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 6:53 pm

Hi,


No. If "git clone" does not work for you, the compilation failed. Do you

run git in-place? Then you _have_ to set


	bindir=$(pwd)


in config.mak. Otherwise, the git wrapper (you wrote "git-clone", but then

you used "git clone") will look in $(HOME)/bin for your executables, and I 


For now, shallow clones only work with the git protocol, i.e. via

git-daemon and ssh.


Hth,

Dscho

-
[view original message]
From: Timur Tabi <timur@...>
To: Johannes Schindelin <Johannes.Schindelin@...>
Cc: Thomas Kolejka <Thomas.Kolejka@...>, Shawn Pearce <spearce@...>, <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Tuesday, November 21, 2006 - 7:12 pm

When I tried that, make spits out this:


GIT_VERSION = 1.4.4.ge1173-dirty

     * new build flags or prefix

(cd perl && /usr/bin/perl Makefile.PL \

         PREFIX='/home/b04825')

Writing Makefile for Git


So I tried this:


make configure

./configure --prefix=$PWD

make


and got this:


     * new build flags or prefix

(cd perl && /usr/bin/perl Makefile.PL \

         PREFIX='/temp/git')

Writing Makefile for Git


which is better, but git-clone still doesn't work:


$ ./git clone --depth 1 git://127.0.0.1/temp/u-boot-83xx

Failed to run command 'clone': Success


$ ./git-clone --depth 1 git://127.0.0.1/temp/u-boot-83xx

usage: git-fetch-pack [--all] [-q] [-v] [-k] [--thin] [--exec=upload-pack]

[host:]directory <refs>...

fetch-pack from 'git://127.0.0.1/temp/u-boot-83xx' failed.


In both of these cases, the git-daemon process doesn't log anything.


I even tried "export GIT_DIR=$PWD", but that didn't do anything.


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Shawn Pearce <spearce@...>
To: Timur Tabi <timur@...>
Cc: <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Wednesday, November 15, 2006 - 6:16 pm

No.  However...


Have you tried "git repack -a -d" to repack the loose objects into

a pack file?  Doing this every so often should reduce your disk

space consumed by a HUGE amount.


--

Shawn.

-
[view original message]
From: Timur Tabi <timur@...>
To: Shawn Pearce <spearce@...>
Cc: <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Wednesday, November 15, 2006 - 6:26 pm

Woah!  It shrunk that bad boy down to 420M!  That should do it, thanks!


I wonder why the powerpc tree shrank so much.  Do you think the maintainer

just needs to run git-repack on his tree?


--

Timur Tabi

Linux Kernel Developer @ Freescale

-
[view original message]
From: Shawn Pearce <spearce@...>
To: Timur Tabi <timur@...>
Cc: <git@...>
Subject: Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?
Date: Wednesday, November 15, 2006 - 6:33 pm

Possible, yes.


However published repositories don't tend to repack as often as

it makes things harder for people who clone/fetch over HTTP rather

than the native git protocol.


The reason is that HTTP can fetch individual loose objects that

you don't have yet, but if the object is only available in a pack

file then you need to fetch the entire pack file.  But you might

already have most of that pack file, so now you are downloading

lots of data you already have.  :-(


--

Shawn.

-
Previous thread: git-blame: handling of revisions and filenames by Alex Riesen on Wednesday, November 15, 2006 - 5:52 pm. (2 messages)

Next thread: Re: Cleaning up git user-interface warts by Shawn Pearce on Wednesday, November 15, 2006 - 7:15 pm. (3 messages)