Re: [PATCH] RFC: git lazy clone proof-of-concept

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Johannes Schindelin <Johannes.Schindelin@...>
Cc: Jakub Narebski <jnareb@...>, Brandon Casey <casey@...>, Nicolas Pitre <nico@...>, Jan Holesovsky <kendy@...>, <git@...>, Junio C Hamano <gitster@...>
Date: Thursday, February 14, 2008 - 7:57 pm

On Thu, Feb 14, 2008 at 05:51:29PM -0600, Brian Downing wrote:

That being said, the most useful output for figuring out where all the
space in the pack is going in my experience is gotten from:

git-verify-pack -v | packinfo.pl -tree -filenames

That will produce a huge amount of output, which is basically the tree
structure of the delta chains in the file.  If things aren't being
deltified together properly, it's usually pretty obvious.

A delta chain in this output looks approximately like this:

#   0   blob 03156f21...     1767     1767 Documentation/git-lost-found.txt @ tags/v1.2.0~142
#   1    blob f52a9d7f...       10     1777 Documentation/git-lost-found.txt @ tags/v1.5.0-rc1~74
#   2     blob a8cc5739...       51     1828 Documentation/git-lost+found.txt @ tags/v0.99.9h^0
#   3      blob 660e90b1...       15     1843 Documentation/git-lost+found.txt @ master~3222^2~2
#   4       blob 0cb8e3bb...       33     1876 Documentation/git-lost+found.txt @ master~3222^2~3
#   2     blob e48607f0...      311     2088 Documentation/git-lost-found.txt @ tags/v1.5.2-rc3~4
#      size: count 6 total 2187 min 10 max 1767 mean 364.50 median 51 std_dev 635.85
# path size: count 6 total 11179 min 1767 max 2088 mean 1863.17 median 1843 std_dev 107.26

# The first number after the sha1 is the object size, the second
# number is the path size.  The statistics are across all objects in
# the previous delta tree.  Obviously they are omitted for trees of
# one object.

# A path size is the sum of the size of the delta chain, including the
# base object.  In other words, it's how many bytes need be read to
# reassemble the file from deltas.

This is also quite slow, as it runs git-ls-tree -t -r on every commit in
the repository to assign file names to blobs.  You can leave out the
-filenames option to not do this (if you don't care about seeing
filenames, that is).

-bcd
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Fri Feb 8, 1:28 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Fri Feb 8, 4:16 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Fri Feb 8, 3:00 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 11:27 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Sun Feb 10, 9:20 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sat Feb 9, 11:10 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 12:43 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 3:50 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brandon Casey, (Thu Feb 14, 3:41 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Thu Feb 14, 4:11 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 3:58 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 2:47 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Tue Feb 12, 4:37 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 3:20 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Fri Feb 15, 5:34 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Thu Feb 14, 4:05 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brandon Casey, (Thu Feb 14, 5:08 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 5:04 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Thu Feb 14, 5:59 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Fri Feb 15, 5:43 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 7:38 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Thu Feb 14, 9:07 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brian Downing, (Thu Feb 14, 7:51 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Thu Feb 14, 8:08 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Thu Feb 14, 9:41 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Shawn O. Pearce, (Sun Feb 17, 4:18 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 17, 2:44 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Junio C Hamano, (Sun Feb 17, 5:05 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Brian Downing, (Thu Feb 14, 7:57 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Thu Feb 14, 4:16 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Tue Feb 12, 5:25 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 5:08 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Tue Feb 12, 5:36 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 5:59 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 6:25 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Tue Feb 12, 6:43 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Linus Torvalds, (Tue Feb 12, 7:39 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Tue Feb 12, 5:05 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 3:42 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Sun Feb 10, 4:11 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Sun Feb 10, 1:01 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 1:36 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 1:22 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Sun Feb 10, 9:42 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 10:04 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jakub Narebski, (Mon Feb 11, 6:11 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Fri Feb 8, 3:26 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Harvey Harrison, (Fri Feb 8, 4:19 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Fri Feb 8, 4:24 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Harvey Harrison, (Fri Feb 8, 4:25 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jon Smirl, (Fri Feb 8, 4:41 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Fri Feb 8, 4:09 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Andreas Ericsson, (Mon Feb 11, 6:13 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Mike Hommey, (Fri Feb 8, 2:49 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 11:06 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Fri Feb 8, 3:04 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Fri Feb 8, 2:20 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Harvey Harrison, (Fri Feb 8, 2:14 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 10:27 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Fri Feb 8, 2:03 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Jan Holesovsky, (Sat Feb 9, 10:25 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Marco Costalba, (Sun Feb 10, 3:23 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 8:08 am)
Re: [PATCH] RFC: git lazy clone proof-of-concept, David Symonds, (Sun Feb 10, 12:46 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 1:45 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sun Feb 10, 3:45 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Johannes Schindelin, (Sun Feb 10, 4:32 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Mike Hommey, (Sat Feb 9, 6:05 pm)
Re: [PATCH] RFC: git lazy clone proof-of-concept, Nicolas Pitre, (Sat Feb 9, 7:38 pm)