I have done some tests about reading git-rev-list output with
different IPC facilities:

- Native Qt QProcess socket based IPC

- pipe based: popen() and fread()

- redirecting git-rev-list to a file (under tmpfs) and 'block reading'
back the file: read() under Qt QFile class. The file, always in
memory, is deleted at the end of loading.


I have tested with different block sizes and different CPU speed on
linux and git trees.

The averaged results on linux tree (about 30MB of data) and CPU set at
1.2GHz are:

- QProcess 6734ms

- pipe and fread() with 64KB blocks 4832ms (38% faster then QProcess)

- temporary file and read() with 64KB blocks 4321ms (10% faster then pipe)


I have not enough knowledge to understand why temporary file is faster
then pipe. My guess is, after reading some docs around, fread() uses a
C standard I/O buffer, while read() is unbuffered.

To make git-rev-list writing a temporary file I create and run a
script more or less like:

git rev-list --header --boundary --parents --topo-order HEAD  >
/tmp/qgit_135902672.txt

There is also some additional kludge to get git-rev-list pid, so to
use in case of a cancel request arrives while loading.


So my request is if it is possible to make git-rev-list write
_directly_ to a file, without shell redirection, I would ask if it is
possible:

- Add a -o, --output option to git-rev-list to specify output file
instead of stdout

- Use an unbuffered binary block write to be as fastest as possible

- Do *not* flush ever the file, it's only a waste under this scenario.


This will let me to avoid the shell launching crap and probably gain
also some more speed.

I understand this could be not exactly a top priority feature for git
people, but I would really like to get the best possible interface
with the plumbing git and the -o options is also a very common one.

Thanks
Marco

P.S: On another thread I explained why I see problematic linking
directly against libgit.a
I rewrite here for completeness:

quoted text
> I've looked again to Shawn idea (and code) of linking qgit
> against libgit.a but I found these two difficult points:
>
> - traverse_commit_list(&revs, show_commit, show_object) is blocking,
> i.e. the GUI will stop responding for few seconds while traversing the
> list. This is easily and transparently solved by the OS scheduler if
> an external process is used for git-rev-list. To solve this in qgit I
> have two ways: 1) call QEventLoop() once in a while from inside
> show_commit()/ show_object() to process pending events  2) Use a
> separate thread (QThread class). The first idea is not nice, the
> second opens a whole a new set of problems and it's a big amount of
> not trivial new code to add.
>
> -  traverse_commit_list() having an internal state it's not
> re-entrant. git-rev-list it's used to load main view data but also
> file history in another tab, and the two calls _could_ be ran
> concurrently. With external process I simply run two instances of
> DataLoader class and consequently two external git-rev-list processes,
> but If I link against libgit.a that would be a big problem.
>
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html