Re: Decompression speed: zip vs lzo

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Nicolas Pitre <nico@...>
Cc: Johannes Schindelin <Johannes.Schindelin@...>, Sam Vilain <sam@...>, Git Mailing List <git@...>, Junio C Hamano <gitster@...>
Date: Thursday, January 10, 2008 - 2:55 am

On Jan 10, 2008 4:41 AM, Nicolas Pitre <nico@cam.org> wrote:

Decompression speed has been shown to be a bottle neck on some tests
involving mainly 'git log'.

Regarding back compatibility I really don't know at what level git
functions actually need to know the compression format, looking at the
code I would say at very low level, functions that deal directly with
inflate() and friends are few [1] and not directly connected to UI,
nor to git config. Is this compression format something user should
know/care? and if yes why?

In my tests the assumption of a source files tar ball is unrealistic,
to test the final size difference I would like testing different
compressions on a big already packaged but still not zipped file.
Someone could be so kind to hint me on how to create such a package
with good quality, i.e. with packaging levels similar to what is done
for public repos?

This does not realistically tests speed because as Junio pointed out
the real decompressing schema is different: many calls on small
objects, not one call on a big one. But if final size is acceptable we
can go on more difficult tests.

Marco

[1] where inflate() is called:

-inflate_it() in builtin-apply.c
-check_pack_inflate() in builtin-pack-objects.c
-get_data() in builtin-unpack-objects.c
-fwrite_sha1_file() in http-push.c and http-walker.c  [mmm interesting
same function in two files, also the signature and the contents seems
the same....]
-unpack_entry_data() in index-pack.c
-unpack_sha1_header(), unpack_sha1_rest(), get_size_from_delta(),
unpack_compressed_entry, write_sha1_from_fd() in sha1_file.c
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Decompression speed: zip vs lzo, Marco Costalba, (Wed Jan 9, 6:01 pm)
Re: Decompression speed: zip vs lzo, Junio C Hamano, (Wed Jan 9, 6:55 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Wed Jan 9, 7:23 pm)
Re: Decompression speed: zip vs lzo, Junio C Hamano, (Wed Jan 9, 7:49 pm)
Re: Decompression speed: zip vs lzo, Johannes Schindelin, (Wed Jan 9, 7:31 pm)
Re: Decompression speed: zip vs lzo, Nicolas Pitre, (Wed Jan 9, 11:41 pm)
Re: Decompression speed: zip vs lzo, Marco Costalba, (Thu Jan 10, 2:55 am)
Re: Decompression speed: zip vs lzo, Dana How, (Thu Jan 10, 3:34 pm)
Re: Decompression speed: zip vs lzo, Marco Costalba, (Thu Jan 10, 7:45 am)
Re: Decompression speed: zip vs lzo, Johannes Schindelin, (Thu Jan 10, 8:12 am)
Re: Decompression speed: zip vs lzo, Marco Costalba, (Thu Jan 10, 8:18 am)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Wed Jan 9, 9:02 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Thu Jan 10, 1:02 am)
Re: Decompression speed: zip vs lzo, Pierre Habouzit, (Thu Jan 10, 5:16 am)
Re: Decompression speed: zip vs lzo, Nicolas Pitre, (Thu Jan 10, 4:39 pm)
Re: Decompression speed: zip vs lzo, Morten Welinder, (Fri Jan 11, 10:18 am)
Re: Decompression speed: zip vs lzo, Pierre Habouzit, (Fri Jan 11, 5:45 am)
Re: Decompression speed: zip vs lzo, Nicolas Pitre, (Fri Jan 11, 10:27 am)
Re: Decompression speed: zip vs lzo, Marco Costalba, (Thu Jan 10, 5:51 pm)
Re: Decompression speed: zip vs lzo, Nicolas Pitre, (Thu Jan 10, 6:18 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Thu Jan 10, 6:01 pm)
Re: Decompression speed: zip vs lzo, Linus Torvalds, (Thu Jan 10, 5:01 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Thu Jan 10, 5:45 pm)
Re: Decompression speed: zip vs lzo, Linus Torvalds, (Thu Jan 10, 6:03 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Thu Jan 10, 6:28 pm)
Re: Decompression speed: zip vs lzo, Linus Torvalds, (Thu Jan 10, 6:56 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Thu Jan 10, 9:01 pm)
Re: Decompression speed: zip vs lzo, Linus Torvalds, (Thu Jan 10, 10:10 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Fri Jan 11, 2:29 am)
Re: Decompression speed: zip vs lzo, Linus Torvalds, (Fri Jan 11, 12:03 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Fri Jan 11, 9:52 pm)
Re: Decompression speed: zip vs lzo, Junio C Hamano, (Sat Jan 12, 12:46 am)
Re: Decompression speed: zip vs lzo, Nicolas Pitre, (Fri Jan 11, 10:32 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Fri Jan 11, 11:06 pm)
Re: Decompression speed: zip vs lzo, Nicolas Pitre, (Sat Jan 12, 12:09 pm)
Re: Decompression speed: zip vs lzo, Johannes Schindelin, (Sat Jan 12, 12:44 pm)
Re: Decompression speed: zip vs lzo, Sam Vilain, (Fri Jan 11, 3:05 am)
Re: Decompression speed: zip vs lzo, Nicolas Pitre, (Thu Jan 10, 5:30 pm)
Re: Decompression speed: zip vs lzo, Pierre Habouzit, (Fri Jan 11, 4:57 am)