On Sat, Sep 06, 2008 at 06:46:29PM -0700, Linus Torvalds wrote:

quoted text
>

>

> On Sat, 6 Sep 2008, Junio C Hamano wrote:

> >

> > This is reproducible  "rev-list --objects --all" in my copy of the kernel

> > repo takes around 47-48 seconds user time, and with the (idiotic) patch it

> > is cut down to 41-42 seconds.

>

> So I had forgotten about that patch since nobody reacted to it.

>

> I think the patch is wrong, please don't apply it, even though it does

> help performance.

>

> The reason?

>

> Right now we depend on "avail_out" also making zlib understand to stop

> looking at the input stream. Sad, but true - we don't know or care about

> the compressed size of the object, only the uncompressed size. So in

> unpack_compressed_entry(), we simply set the output length, and expect

> zlib to stop when it's sufficient.

>

> Which it does - but the patch kind of violates that whole design.

>

> Now, it so happens that things seem to work, probably because the zlib

> format does have enough synchronization in it to not try to continue past

> the end _anyway_, but I think this makes the patch be of debatable value.

>

> I'm starting to hate zlib. I actually spent almost a week trying to clean

> up the zlib source code and make it something that gcc can compile into

> clean code, but the fact is, zlib isn't amenable to that. The whole "shift

>  bits in from the buffer" approach means that there is no way to make

> zlib generate good code unless you are an insanely competent assembly

> hacker or have tons of registers to keep all the temporaries live in.

>

> Now, I still do think that all my reasons for choosing zlib were pretty

> solid (it's a well-tested piece of code and it is _everywhere_ and easy to

> use), but boy do I wish there had been alternatives. 

I know at least 7-zip has its own gzip compression/decompression code

(though it's C++). Maybe some other tools have theirs too.
Anyways, if it can make a speed difference, it might be worth having a

minimalist custom gzip compression/decompression "library" embedded

withing git.
Mike

--

To unsubscribe from this list: send the line "unsubscribe git" in

the body of a message to majordomo@vger.kernel.org

More majordomo info at  http://vger.kernel.org/majordomo-info.html