Re: git-import.sh using git-fast-import

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Shawn O. Pearce
Date: Tuesday, August 28, 2007 - 8:47 pm

Jeff King <peff@peff.net> wrote:

It may also have to do with the large amount of data going over a
pipe, as well as the fact that we have to write the data into the
packfile then go back and reread all of it to compute the SHA-1 of
the entire thing.  That's a lot of IO, all going through your poor
little kernel.  ;-)


Yea, so like the parallel pack-objects experiment that Nico had
done recently we decreased wall-clock time at the expense of using a
larger amount of the system resources.  That is bad as we use more
CPU time than we saved in wallclock time.  Usually a bad tradeoff,
unless you have a realtime requirement you have to meet.


Thanks for running this, it was interesting to see.  There are a
lot of ways that fast-import could be improved to be faster-import
(heh) but we haven't tried to do anything as it has typically been
fast enough.  Some of the ideas have been discussed on list, but
they just haven't been implemented.  One problem is fast-import
is O(n^2 + n log n) in updating its internal tree state.  Not very
fast on trees with a lot of entries in them.  At least fast-import
doesn't use a flat structure like the index.

-- 
Shawn.
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
git-import.sh using git-fast-import, Nguyen Thai Ngoc Duy, (Mon Aug 27, 9:13 am)
Re: git-import.sh using git-fast-import, Johannes Gilger, (Mon Aug 27, 10:44 am)
Re: git-import.sh using git-fast-import, Shawn O. Pearce, (Mon Aug 27, 8:54 pm)
Re: git-import.sh using git-fast-import, Nguyen Thai Ngoc Duy, (Mon Aug 27, 9:04 pm)
Re: git-import.sh using git-fast-import, Nguyen Thai Ngoc Duy, (Mon Aug 27, 9:14 pm)
Re: git-import.sh using git-fast-import, Shawn O. Pearce, (Mon Aug 27, 9:20 pm)
Re: git-import.sh using git-fast-import, Shawn O. Pearce, (Mon Aug 27, 9:22 pm)
Re: git-import.sh using git-fast-import, Jeff King, (Mon Aug 27, 10:54 pm)
Re: git-import.sh using git-fast-import, Shawn O. Pearce, (Mon Aug 27, 11:11 pm)
Re: git-import.sh using git-fast-import, Shawn O. Pearce, (Mon Aug 27, 11:21 pm)
Re: git-import.sh using git-fast-import, Nguyen Thai Ngoc Duy, (Mon Aug 27, 11:26 pm)
Re: git-import.sh using git-fast-import, Jeff King, (Mon Aug 27, 11:33 pm)
Re: git-import.sh using git-fast-import, Jeff King, (Mon Aug 27, 11:36 pm)
Re: git-import.sh using git-fast-import, Jeff King, (Mon Aug 27, 11:43 pm)
Re: git-import.sh using git-fast-import, Shawn O. Pearce, (Tue Aug 28, 8:47 pm)
Re: git-import.sh using git-fast-import, Jeff King, (Tue Aug 28, 9:11 pm)
Re: git-import.sh using git-fast-import, Nicolas Pitre, (Wed Aug 29, 8:20 am)