Re: git-feed-mail-list.sh

Previous thread: Re: [ANNOUNCE] Git wiki by Jakub Narebski on Wednesday, May 3, 2006 - 9:46 am. (1 message)

Next thread: What's in git.git by Junio C Hamano on Wednesday, May 3, 2006 - 11:54 am. (1 message)
From: David Woodhouse
Date: Wednesday, May 3, 2006 - 10:48 am

I've just updated the script which feeds the
git-commits-{head,24}@vger.kernel.org lists -- it now looks like this...

- Git-only; no longer uses cg-*
- No longer uses its own clone repo -- works directly from the original
- Handles MIME encoding of Subject: headers when necessary
  (although this is a bit icky in shell; especially _my_ shell).

Should I be using git-format-patch for this?

-- 
dwmw2
From: Junio C Hamano
Date: Wednesday, May 3, 2006 - 9:35 pm

If you were to do bashism local, don't you want to also localize
other variables like key, SUBHEX, NEWSUB,...?

It may make sense to enhance format-patch to do the Q encoding,

With GIT 1.3.0 and later:

	git diff --patch-with-stat $parent..$commit



I am not sure if earlier base=$(git-rev-parse $1) is needed if
you do this here...

-

From: Junio C Hamano
Date: Wednesday, May 3, 2006 - 10:16 pm

Or at least lose "diffstat -p1" and replace it with

	git-apply --stat --status


As I wrote it this is broken, sorry.

This assumes you stop doing "MAILTAG=.git/refs/tags/MailDone"
by hand and lose "do we have GIT_DIR" logic as well.
Instead define MAILTAG=tags/MailDone or maybe refs/tags/MailDone
and let "git rev-parse --default refs/tags/MailDone" figure out
what to do when GIT_DIR is set or unset.

-

From: David Woodhouse
Date: Monday, May 8, 2006 - 4:44 pm

Yes, that would be useful. We should perhaps to the From: and To:
headers too. Here's my current version (thanks for the feedback)...

The remaining problem is that the invocation of 'date' doesn't work with
new versions of coreutils. This...

   date=(${rest#*> })
   sec=${date[0]}; tz=${date[1]}
   dtz=${tz/+/+ }; dtz=${dtz/-/- }
   pdate="$(date -Rud "1970-01-01 UTC + $sec sec $dtz" 2>/dev/null)"

... doesn't work any more on FC-5, because:

 $ date -Rud '1970-01-01 UTC + 1147104611 sec + 0100'
date: invalid date `1970-01-01 UTC + 1147104611 sec + 0100'

-- 
dwmw2
From: Linus Torvalds
Date: Monday, May 8, 2006 - 5:19 pm

Well, you might choose to just not use "git-cat-file commit" but instead 
ask git to format the thing for you.

Ie you could probably more easily parse the data from something like

	git show -B --patch-with-stat --pretty=fuller $commit

instead of using "git-cat-file commit $commit" and generating the stat and 
diff manually.

That way you get the dates etc pretty-printed for you by git.

			Linus
-

From: David Woodhouse
Date: Monday, May 8, 2006 - 5:28 pm

Aha, thanks. Git has learned to do a lot more since I first started
hacking up a copy of git-log.sh to feed the mailing lists, and it even
had to walk the commit tree manually :)

The output of (the undocumented) '--pretty=fuller' is probably good
enough that I can just feed the mailing list with it directly. I think I
have to add the commit and the parent manually, but that's easy enough
to do -- the commit is obviously known, and the parent is just
$(git-rev-parse $commit^1).

Anyone got any objections to switching the kernel git-commits-* lists to
this format?

-- 
dwmw2

-

From: Junio C Hamano
Date: Monday, May 8, 2006 - 5:36 pm

If you are going to parse it anyway, this would probably be easier.

	$ git show --patch-with-stat --pretty=fuller --parents "$commit"

Why would you want -B, by the way?

-

From: Bertrand Jacquin
Date: Monday, May 8, 2006 - 5:53 pm

What is --parents for ?

Is there a way to have a --pretty=fuller with date append to Author
and Commiter name and email instead of have 1 lign after each ?

--
Beber
#e.fr@freenode
-

From: Junio C Hamano
Date: Monday, May 8, 2006 - 5:57 pm

In the part you did not quote, David wanetd a way to grab the

sed is your friend.

-

From: Bertrand Jacquin
Date: Monday, May 8, 2006 - 5:59 pm

Ok, sorry. I read too fast.

--
Beber
#e.fr@freenode
-

From: Linus Torvalds
Date: Monday, May 8, 2006 - 5:45 pm

As long as the "commit <sha1>" id is there (and "--pretty=fuller" does 
have it), I'll be happy. At some point, the commit mailing list didn't 
actually mention the commit ID itself, just the tree/parent IDs.

The "fuller" format should be fine, if you care about committer. Otherwise 
just use the standard "--pretty", which drops committer info.

		Linus
-

From: David Woodhouse
Date: Monday, May 8, 2006 - 6:01 pm

Ah, right. Those are _commit_ IDs in that strange first line. I'll
reformat those to 'Commit:' and 'Parent:' for the mailing list.

Having 'git-show --pretty=email' would be nice. I think Junio is working

Yeah, I know -- and I got complaints :)

-- 
dwmw2

-

From: Junio C Hamano
Date: Monday, May 8, 2006 - 6:05 pm

That's the replacement of "git format-patch".  If you have a
chance, please try out what's in "next".  Johannes did quite a
nice enhancements.

-

From: David Woodhouse
Date: Monday, May 8, 2006 - 6:12 pm

I think I'd best wait for it to turn up in the release; preferably
already capable of MIME quoting. I don't like touching the scripts which
feed the mailing lists :)

This is what I currently have... since it changes the format I think
I'll won't deploy it yet -- I'll wait until the "replacement of
git-format-patch" is done, in case that would change the format _again_.
I'd rather not change the format I send to the list twice within a few
weeks.

-- 
dwmw2
From: Martin Mares
Date: Wednesday, May 10, 2006 - 1:49 am

Wouldn't it be easier to feed the output to a MUA and letting it handle
the MIME stuff for you?

I am using mutt for this purpose:

mutt -x -e 'set charset="utf-8"; set send_charset="us-ascii:iso-8859-2:utf-8"' -s "$subj" "$recipient" <$out

				Have a nice fortnight
-- 
Martin `MJ' Mares   <mj@ucw.cz>   http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
Next lecture on time travel will be held on previous Monday.
-

From: Linus Torvalds
Date: Monday, May 8, 2006 - 6:27 pm

Right. That first line (that starts with "commit") lists the commit ID, 
and if you say "--parents", the commit ID's of the parents will be 
appended.

So if you want to turn that into "Commit: <id>" and "Parent: <id>", you'll 
want to do something like this:

  git show --no-abbrev -C --patch-with-stat --pretty=fuller --parents $commit |
	sed '1 s/commit \([0-9a-f]*\)/Commit:     \1\nParent:    /'

which should look pretty (count the spaces to make sure it lines up 
right with the other fields).

(And if you ever want to report on merges, you'll want to change that a 
bit, but it should be reasonably close to the above)

		Linus
-

From: Bertrand Jacquin
Date: Monday, May 8, 2006 - 5:55 pm

Is there a way to track merge like that ? Documentation is not very
clear and near from empty.

--
Beber
#e.fr@freenode
-

From: Junio C Hamano
Date: Monday, May 8, 2006 - 6:03 pm

The command line you quoted shows the relevant information for
people who want to know what happened in that merge.

Namely:

 * it always shows the header and the message

 * it shows the changes that are not trivial (i.e. merge parents
   have overlapping different versions and manual resolution
   resulted in something different from either parents).

It is not a replacement for format-patch, but I think the commit
mailing list is not for machines to receive and apply the
received patches, but for humans to inspect, so it would be more
suitable than a naive alternative of showing diff from all
parents concatenated together.

-

From: Bertrand Jacquin
Date: Monday, May 8, 2006 - 6:09 pm

That's right. And don't want to do that.

But I would like to send an email after merge to inform people that:

o tree ``a'' and ``b'' have been merged.
o made by John Doe at a time
o show a diffstat.
o show a --short-log=oneline from merge base.

--
Beber
#e.fr@freenode
-

From: Junio C Hamano
Date: Monday, May 8, 2006 - 7:41 pm

Forgetting about a fast-forward merge,  (1) and (2) are
available in the commit header and the commit log, so is (4) if
you enable merge.summary configuration like Linus does in his
kernel repository.

The comment on diffstat Linus already made applies to (3), but
if you want you could do "git diff --stat HEAD^..HEAD" to see
what happened to that branch by merging the other branch into
it.

-

From: Linus Torvalds
Date: Monday, May 8, 2006 - 8:06 pm

NOTE! Please don't enable "merge.summary" if you ever merge from the 
upstream tree. That just looks ugly. Your merge messages will be just 
filled with crap that has nothing to do with your tree - and everything to 
do with all the _unrelated_ normal development that happened in the tree.

So in general, "merge.summary" makes sense only for trees that pull from 
downstreams, and never merge with anything upstream. My tree obviously 
does that for the kernel. Think of it as a "top-level maintainer" flag, 
although it works find also for sub-maintainers as long as they 
synchronize upwards _purely_ by being pulled from, not by pulling.

But if you want to get it for any random merges, you can always just do

	git log -11 --pretty=oneline ^$commit^ $commit^@ |
		sed 's/[0-9a-f]* // ; 11 s/.*/\.\.\./' 

which will show up to the ten first commits that were merged (and turn the 
eleventh one, if it exists, into "..." - that's a pretty disgusting trick 
to make it show when you left things out).

That "^$commit^ $commit^@" part is important. It may look like some 
deranged git smiley, but it does exactly what you want it to do: take all 
the parents of the commit, but ignore any commit reachable from the first 
one (the "mainline" of the person who did the commit).

The ^@ syntax is obviously pretty new, so it requires a modern git.

		Linus
-

From: Junio C Hamano
Date: Tuesday, May 9, 2006 - 12:32 am

It is indeed very quite new.  Merged into "master" branch at the
beginning of this month.

I often wish we had a straightforward way to tell when a given
feature went into the mainline, not just appeared on a topic
branch.  In this case, I said:

	$ git whatchanged -p -S'"^@"' master -- revision.c

to find ea4a19 commit (Apr 30 00:54:29 2006 -0700).  But that
was when the feature was first made on one of my topic branches,
which is not what I was looking for.

By looking at gitk, I can then tell 83262e (May 1 01:54:27)
merged it to "next", and 746437 (May 1 22:55:40) merged it to
"master".

In general this is an unsolvable question, because I can have a
topic branch forked off of the tip of "master", cook it for a
few days without advancing "master" at all, and merge it to
"master" after that.  But such a merge will be a fast-forward.




-

From: Linus Torvalds
Date: Monday, May 8, 2006 - 6:18 pm

Sure.

If you want to track merges and get their patches, add the "--cc" flag, 
which tells git to use the "conflict combination patch" that shows any 
visible conflicts.

(NOTE NOTE NOTE! This is _not_ the same as showing what conflicted: if you 
edited the result to match one of the original branches, it will be quiet 
in --cc, but if the result of a conflict was something that was in 
_neither_ branch, it will be shown! So most clean merges will not show any 
conflict diff at all, but the diffstat will be shown against the "first 
parent").

And you probably don't want to abbreviate the parent commit SHA1's (which 
are shown for merges, but not regular commits), so add "--no-abbrev".

If you want to show parents for _all_ commits, you could do something like

  git show --no-abbrev --cc -C --patch-with-stat --pretty=fuller --parents |
	sed '1 s/commit [0-9a-f]*/\0\nParents:   / ; /^Merge: / d'

which removes a potential "Merge: " line in favour of listing the parents 
on a "Parents:" line, and which also shows merges nicely.

That said, the diffstat for merges is usually just a lot of noise. It's 
sometimes nice (you've merged from a topic branch), but if you have merged 
from the mainline _into_ a topic branch, it's just annoying.

So the above is just a wild suggestion. Caveat emptor.

		Linus
-

From: Junio C Hamano
Date: Tuesday, May 9, 2006 - 12:15 am

True.  We made --cc --patch-with-stat to do the combined patch