Re: git pack/unpack over bittorrent - works!

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Ted Ts'o
Date: Saturday, September 4, 2010 - 11:14 am

On Sat, Sep 04, 2010 at 03:50:29PM +0100, Luke Kenneth Casson Leighton wrote:

I ask the question because I think being clear about what goals might
be are critically important.  If in fact the goals is to evade
detection be spreading out responsibility for code which is illegal in
some jurisdictions (even if they are commonly used and approved of by
people who aren't spending millions of dollars purchasing
congresscritters), there are many additional requirements that are
imposed on such a system.

If the goal is speeding up git downloads, then we need to be careful
about exactly what problem we are trying to solve.


Maybe it's because the benefits don't exist for many people?  At least
where I live, my local ISP (Comcast, which is very common internet
provider in the States) deliberately degrades the transfer of
peer2peer downloads.  As a result, it doesn't make sense for me to use
bittorrent to download the latest Ubuntu or Fedora iso image.  It's in
fact much faster for me to download it from an ftp site or a web site.

And git is *extremely* efficient about its network usage, since it
sends compressed deltas --- especially if you already have a base
responsitory estlablished.  For example, I took a git repository which
I haven't touched since August 4th --- exactly one month ago --- and
did a "git fetch" to bring it up to date by downloading from
git.kernel.org.  How much network traffic was required, after being
one month behind?  2.8MB of bytes received, 133k of bytes transmitted.

That's not a lot.  And it's well within the capabilities of even a
really busy server to handle.  Remember, peer2peer only helps if the
aggregate network bandwidth of the peers is greater than (a) your
download pipe, or (b) a central server's upload pipe.  And if we're
only transmitting 2.8MB, and a git.kernel.org has an aggregate
connection of over a gigabit per second to the internet --- it's not
likely that peer2peer would in fact result in a faster download.  Nor
is it likely that that git updates are likely to be something which
the kernel.org folks would even notice as a sizeable percentage of
their usable network bandwidth.  First of all, ISO image files are
much bigger, and secondly, there are many more users downloading ISO
files than there are developers downloading git updates, and certainly
relatively few developers downloading full git repositories (since
everybody genreally tries really hard to only do this once).


Well, perhaps it's because what we have right now works pretty well.   :-)

Which brings me back to my original question --- what problem exactly
are you trying to solve?  What's the scenario?

If the answer is "peer2peer is cool technology, and we want to play",
that's fine.  Put it would confirm the hypothesis that in this case,
peer2peer is a solution looking for a problem...

					- Ted
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Wed Sep 1, 7:36 am)
Re: git pack/unpack over bittorrent - works!, Nguyen Thai Ngoc Duy, (Wed Sep 1, 3:04 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 6:37 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 6:53 am)
Re: git pack/unpack over bittorrent - works!, Ævar Arnfjörð Bjarmason, (Thu Sep 2, 7:08 am)
Re: git pack/unpack over bittorrent - works!, A Large Angry SCM, (Thu Sep 2, 8:33 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 8:42 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 8:51 am)
Re: git pack/unpack over bittorrent - works!, Jeff King, (Thu Sep 2, 8:58 am)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Thu Sep 2, 9:41 am)
Re: git pack/unpack over bittorrent - works!, A Large Angry SCM, (Thu Sep 2, 9:58 am)
Re: git pack/unpack over bittorrent - works!, A Large Angry SCM, (Thu Sep 2, 10:06 am)
Re: git pack/unpack over bittorrent - works!, A Large Angry SCM, (Thu Sep 2, 10:09 am)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Thu Sep 2, 10:21 am)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Thu Sep 2, 10:31 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 11:07 am)
Re: git pack/unpack over bittorrent - works!, Casey Dahlin, (Thu Sep 2, 11:23 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 12:17 pm)
Re: git pack/unpack over bittorrent - works!, Shawn O. Pearce, (Thu Sep 2, 12:29 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 12:41 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 12:51 pm)
Re: git pack/unpack over bittorrent - works!, A Large Angry SCM, (Thu Sep 2, 12:52 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 1:06 pm)
Re: git pack/unpack over bittorrent - works!, Brandon Casey, (Thu Sep 2, 1:28 pm)
Re: git pack/unpack over bittorrent - works!, Jakub Narebski, (Thu Sep 2, 1:45 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 1:48 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 2:10 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Thu Sep 2, 2:19 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Thu Sep 2, 4:09 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Thu Sep 2, 5:29 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Thu Sep 2, 5:36 pm)
Re: git pack/unpack over bittorrent - works!, Nguyen Thai Ngoc Duy, (Thu Sep 2, 7:48 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Fri Sep 3, 3:23 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Fri Sep 3, 3:34 am)
Re: git pack/unpack over bittorrent - works!, Theodore Tso, (Fri Sep 3, 3:37 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Fri Sep 3, 3:54 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Fri Sep 3, 3:55 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Fri Sep 3, 4:04 am)
Re: git pack/unpack over bittorrent - works!, Junio C Hamano, (Fri Sep 3, 10:03 am)
Re: git pack/unpack over bittorrent - works!, Junio C Hamano, (Fri Sep 3, 10:12 am)
Re: git pack/unpack over bittorrent - works!, Ted Ts'o, (Fri Sep 3, 11:31 am)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Fri Sep 3, 12:41 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Fri Sep 3, 2:11 pm)
Re: git pack/unpack over bittorrent - works!, Nguyen Thai Ngoc Duy, (Fri Sep 3, 5:24 pm)
Re: git pack/unpack over bittorrent - works!, Nguyen Thai Ngoc Duy, (Fri Sep 3, 5:57 pm)
Re: git pack/unpack over bittorrent - works!, Artur Skawina, (Fri Sep 3, 6:52 pm)
Re: git pack/unpack over bittorrent - works!, Theodore Tso, (Fri Sep 3, 6:57 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Fri Sep 3, 9:39 pm)
Re: git pack/unpack over bittorrent - works!, Kyle Moffett, (Fri Sep 3, 10:23 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Fri Sep 3, 10:40 pm)
Re: git pack/unpack over bittorrent - works!, Artur Skawina, (Fri Sep 3, 10:42 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Fri Sep 3, 11:13 pm)
Re: git pack/unpack over bittorrent - works!, Theodore Tso, (Sat Sep 4, 4:46 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 4:58 am)
Re: git pack/unpack over bittorrent - works!, Theodore Tso, (Sat Sep 4, 5:00 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 5:33 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 5:44 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 6:14 am)
Re: git pack/unpack over bittorrent - works!, Artur Skawina, (Sat Sep 4, 6:42 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 7:06 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 7:50 am)
Re: git pack/unpack over bittorrent - works!, Artur Skawina, (Sat Sep 4, 10:23 am)
Re: git pack/unpack over bittorrent - works!, Ted Ts'o, (Sat Sep 4, 11:14 am)
Re: git pack/unpack over bittorrent - works!, Artur Skawina, (Sat Sep 4, 11:46 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 1:00 pm)
Re: git pack/unpack over bittorrent - works!, Jakub Narebski, (Sat Sep 4, 1:20 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 1:47 pm)
Re: git pack/unpack over bittorrent - works!, Jakub Narebski, (Sat Sep 4, 2:16 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sat Sep 4, 2:24 pm)
Re: git pack/unpack over bittorrent - works!, Ted Ts'o, (Sat Sep 4, 3:41 pm)
Re: git pack/unpack over bittorrent - works!, Ted Ts'o, (Sat Sep 4, 3:47 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Sat Sep 4, 6:18 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Sat Sep 4, 6:32 pm)
Re: git pack/unpack over bittorrent - works!, Tomas Carnecky, (Sat Sep 4, 6:43 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Sat Sep 4, 7:16 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sun Sep 5, 10:16 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sun Sep 5, 10:22 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sun Sep 5, 10:25 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Sun Sep 5, 11:05 am)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Sun Sep 5, 4:52 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Sun Sep 5, 5:05 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Mon Sep 6, 6:23 am)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Mon Sep 6, 9:51 am)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Mon Sep 6, 3:33 pm)
Re: git pack/unpack over bittorrent - works!, Junio C Hamano, (Mon Sep 6, 4:34 pm)
Re: git pack/unpack over bittorrent - works!, Nicolas Pitre, (Mon Sep 6, 4:57 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Mon Sep 6, 5:17 pm)
Re: git pack/unpack over bittorrent - works!, Luke Kenneth Casson ..., (Mon Sep 6, 5:29 pm)