On Jan 22, 2008, at 9:46 PM, Junio C Hamano wrote:

quoted text
> Kevin Ballard <kevin@sb.org> writes:
>
>> I just glanced at git-filter-branch.sh (and I must say I was
>> incredibly surprised to find out it was a shell script) and it seems
>> it never runs git-gc or git-repack. Doesn't that end up with the same
>> problems as git-svn sans git-repack when filtering a large number of
>> commits? I was just thinking, if I were to git-filter-branch on my
>> massive repo (in fact, the same repo that started this thread, with
>> over 33000 commits in the upstream svn repo), even if I just do
>> something as simple as change the commit msg wont I end up with
>> thousands of unreachable objects? I shudder to think how many
>> unreachable objects I would have if I pruned the entire dports
>> directory off of the tree.
>>
>> Am I missing something, or does git-filter-branch really not do any
>> garbage collection? I tried reading the source, but complex bash
>> scripts are almost as bad as perl in terms of readability.
>
> Theoretically yes, and it largely depends on what you do, but
> filter-branch goes over the objects that already exists in your
> repository, and hopefully you won't be rewriting majority of
> them.
>
> So the impact of not repacking is probably much less painful in
> practice.
>
> But again as I said, it largely depends on what you do in your
> filter.  If you are upcasing (or convert to NFD ;-)) the
> contents of all of your blob objects, you would certainly want
> to repack every once in a while.


I'm actually considering what the cost would be of switching macports  
to git (not that it will ever happen - too many anonymous people pull  
from svn trunk). Right now the svn trunk contains a subfolder for the  
source code and another subfolder for all ~4400+ Portfiles. In such a  
theoretical move, I'd want to split that up, probably into two  
unrelated branches. Doing so would mean running git-filter-branch over  
a linear commit history that's 31580 objects long, with a tree filter  
to prune the dports directory away and a msg filter to remove the svn- 
id stuff that git-svn left behind. This means that every single commit  
objects would be changed, as well as the root tree object for every  
single commit. That would be about 63160 objects. I'd also have to  
figure out some way to remove the commit objects entirely that only  
reference the dports directory. Then I'd have to do it again with the  
opposite tree filter (to prune everything but the dports directory and  
move the contents of the dports directory up one level) and same msg  
filter. Granted, if I do the first action in a branch, that leaves no  
unreachable objects (since the originals are still referenced), but  
the second operation definitely would leave unreachable objects, and  
were I to clone the repository instead and do the operations in the  
different repos (which is perfectly legitimate - otherwise I'd have to  
clone it after everything else and then delete branches) then both  
actions would leave thousands of objects unreachable.

I'd suggest a patch to run git gc --auto, but it looks like you just  
did in a subsequent email. As for your comments about the reflogs,  
can't I disable recording those, at least temporarily? I'd rather  
clean up after myself as I work rather than balloon the repository and  
collapse it in a single operation at the end.

-Kevin Ballard

-- 
Kevin Ballard
http://kevin.sb.org
kevin@sb.org
http://www.tildesoft.com