Re: [ANNOUNCE] pg - A patch porcelain for GIT

Previous thread: [PATCH] delta micro optimization by Nicolas Pitre on Friday, February 10, 2006 - 11:42 am. (3 messages)

Next thread: [PATCH] Ignore commits for which cvsps can't identify a branch by Christian Biesinger on Friday, February 10, 2006 - 2:02 pm. (6 messages)
From: Shawn Pearce
Date: Friday, February 10, 2006 - 12:59 pm

I just posted the first public version of pg, a GIT porcelain for
managing patches.  Think StGIT, but better in some ways:

Feature Summary:

- Maximum compatibility with other GIT porcelains.

    pg was designed to interoperate with core GIT and the other
    GIT porcelains as much as possible.  GIT favorites like git-am
    can be used to modify a pg managed patch, and vice-versa,
    and without requiring changes to the other GIT tools.

- Simplified command line user interface.

    pg tries to simplify GIT by 'hiding' the index and behaving like
    more traditional SCMs which only look at `HEAD` (last commit)
    and the working directory (files).

- Preserves change history of patches.

    The complete change history associated with each patch is
    maintained directly within GIT.  By storing the evolution of a
    patch as a sequence of GIT commits standard GIT history tools
    such as gitk can be used.

- Its prune proof.

    The metadata structure is stored entirely within the refs
    directory and the object database, which means you can safely use
    git-prune without damaging your work, even for unapplied patches.

- Preserves patch series during clone.

    The metadata structure used by pg allows git-clone to preserve
    the patch series information, without changes required to
    git-clone.  (Patch series information is not preserved during
    git-pull/git-push however.)

- Mix and matching of changes (bug fixes/features).

    By maintaining changes as individual patches it is possible to
    apply individual changes to the current working directory and
    to unapply them just as easily.

- Automatic detection (and cancellation) of returning patches.

    pg automatically detects when a patch is received from
    the upstream GIT repository during a pg-rebase and deletes
    (cancels) the local version of the patch from the patch series.
    The automatic cancelling makes it easy to use pg to track and
    develop changes on top of ...
From: Greg KH
Date: Friday, February 10, 2006 - 1:41 pm

Hm, is there any way to import an existing patch into pg?

thanks,

greg k-h
-

From: Shawn Pearce
Date: Friday, February 10, 2006 - 2:04 pm

Doh!  I haven't needed to do that yet.  I'll code up a pg-import
later tonight.  But since git and pg play nice together you can
do this:

	pg-new Patch-Name
	git-apply the-patch-file.patch
	pg-ci -m"Importing the-patch-file.patch..."

or even:

	pg-new Patch-Name
	git-am mbox

and keep the 'history' stored in the mailbox.

So pg-import won't amount to a very long script.  :-|
-

From: Greg KH
Date: Friday, February 10, 2006 - 4:20 pm

well, as my quilt tree is around 200 patches right now, that would be
annoying to have to do by hand :)

thanks,

greg k-h
-

From: Petr Baudis
Date: Friday, February 10, 2006 - 2:17 pm

Hi,

Dear diary, on Fri, Feb 10, 2006 at 08:59:14PM CET, I got a letter

  it sounds interesting. I've been thinking about wrapping some patch
queue tool in Cogito (post-1.0) and pg might be a better choice than
StGIT.

  One thing I dislike on both StGIT and pg is that they both try to
build a full-fledged porcelain on top of GIT, instead of just focusing
on the patch management, doing it well and providing a convenient user
interface (well, can't say about pg's interface, didn't try it yet).
Instead of having pg-add, pg-log, or pg-status it might be more fruitful

But while it claims to be compatible with all the porcelains, it at
least cannot be clone by them. ;) The GIT repository is not quite a
valid GIT repository since it is missing the HEAD and Cogito clones
based on this file instead of just assuming that your head is on the
master branch.


Also, when cloning it gives me a little unnerving errors like

error: File 6427c0154400f578d9cdff178e01e946db6f714f
(http://www.spearce.org/projects/scm/pg.git/objects/64/27c0154400f578d9cdff178e01e946db...)
corrupt

(but strangely, fsck-objects later does not complain).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams
-

From: Shawn Pearce
Date: Friday, February 10, 2006 - 2:38 pm

Very valid points.  Before writing pg I used Cogito exclusively
and found git-core too cumbersome to use directly.  When I started
writing pg I didn't intend on replacing everything GIT and Cogito
offers; I was trying to only create the patch stack but still use
the Cogito tools for everything else.

But two things happened:

  1) Cogito didn't run well on a Solaris box I wanted to try and
     use it in; apparently we don't have enough GNU shell commands
     available and Cogito fell over.  (But right now I'd bet pg
     will behave the same if not worse. I haven't had time to try
     it. *sigh*)

  2) I found myself suddenly typing 'pg-log' and 'pg-diff' rather
     than 'git-log' and 'git-diff'.  Call it future muscle memory?
     I hadn't written either of these scripts so I was getting a lot
     of '-bash: pg-log: command not found' errors from my shell.
     So they both became 1 line wrappers around the git-core
     versions, just to save my sanity.

I would agree with trying to integrate some of the workflow idealogy
presented by StGIT and pg into something more mainstream such as
git-core or Cogito.  Right now I'm using pg as a proving ground to
feel out how some of that might work in one particular environment:

  A development team I work with is stuck using PVCS Version
  Manager 6.  Moving source code from a developer to a tester is a
  huge nightmare; not only must the developer check the code into
  the version control system but he/she must also write a bug report
  in a bug database to tell someone else to get the source file
  and give it to the tester.  Its a horrible workflow.  GIT + pg +
  additional custom scripts seems to be easing the pain somewhat;

Fixed.  That's my fault - my hosting provider doesn't have GIT
installed and thus I had to publish my repository over rsync+ssh.
But git-push doesn't support that protocol type anymore.  :-| So
I packed everything into pack files, pruned the object directory,

I've seen the same.  I think it is ...
From: Petr Baudis
Date: Friday, February 10, 2006 - 2:47 pm

Dear diary, on Fri, Feb 10, 2006 at 10:38:18PM CET, I got a letter

I'm always listening for bugreports. Besides requiring bash, Cogito _is_

I see. IIRC Catalin gave the similar reasoning. (Obviously, my


On a second thought, this is probably simply caused by the web server
not reporting 404 on missing files.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams
-

From: Junio C Hamano
Date: Friday, February 10, 2006 - 3:07 pm

I suspect that is it.  The webserver is broken^W not being a
good network citizen.

-

From: Petr Baudis
Date: Monday, February 13, 2006 - 2:00 pm

Hi,

Dear diary, on Fri, Feb 10, 2006 at 10:17:40PM CET, I got a letter

  so I've used it a bit and I'm going back to StGIT, at least for now.
It is not really usable for me so far, since it is missing two crucial
things:

	* Patch description tracking. Patch description is almost as
	important as patch contents for me, and pg just doesn't track it
	for now.  It would be best if it just seeded the patch
	description by the first commit message and then allow you edit
	it at the refresh time.

	* Mail interface. StGIT can pre-fill the patch description with
	my signoff line, but more importantly when I write

		stg mail patchname

	it will mail the patch to the addresses I configured it to,
	prepend [PATCH] to the subject line and stuff.

  So, my patchqueue workflow is "I do some random third-party patches
for some software and want to manage, update, and submit them easily."
PG does not make it much easier now, unfortunately.

  Some common gripes for both StGIT and pg (well, I'm using some
ridiculously old StGIT version, so this may not apply anymore there):

	* stg new --force - seriously, what's the point?! I always to
	the change first and when it's any good, I want to create a
	patch for it.

	* I can't just get the patch in its "canonical ready-to-mail
	form" on stdout so that I could easily review it. Why is
	pg-export insisting to dump it to a file?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams
-

From: Catalin Marinas
Date: Tuesday, February 14, 2006 - 2:26 am

This was fixed couple of weeks ago in the main branch. No need to pass

To view the patch you can use 'stg diff -r <patch>/' but it doesn't
show the description. Dumping the full patch on stdout would be
useful, indeed. The export and mail commands use different templates
and the latter even adds the standard mail headers. Which of these two
commands would you prefer to dump the patch on stdout (both is fine as
well)?

Another thing that's missing in StGIT is the import of a series of
patches. At the moment I run a small shell script to import individual
patches.

-- 
Catalin
-

From: Karl
Date: Tuesday, February 14, 2006 - 3:08 am

One thing I would like to see in stgit is the opposite of "stg
commit"; instead of converting patches to regular commits, take the
topmost regular commits and convert them to patches.

For example, "stg uncommit foo bar baz" would -- regardless of any
existing patches, applied or not -- convert the top three regular
commits, with comments and all, to stgit patches called foo, bar, and
baz. These would be already applied, at the bottom of the stack. I
imagine all one would have to do is to modify some stgit metadata, so
the operation could be really cheap.

Of course, "stg uncommit" is allowed to reject any commit with more
than one parent, since those can't be represented as stgit patches.

This would perhaps not add much power to an all-stgit workflow, but it
would be a really convenient way to edit recent git history. Sort of
like a more convenient rebase. And a great way to lure new users. :-)

-- 
Karl Hasselstr
From: Chuck Lever
Date: Tuesday, February 14, 2006 - 8:22 am

i think you want "stg pick --reverse" ?
From: Karl
Date: Tuesday, February 14, 2006 - 9:07 am

On 2006-02-14 10:22:51 -0500, Chuck Lever wrote:

> Karl Hasselstr
From: Chuck Lever
Date: Tuesday, February 14, 2006 - 1:58 pm

gotcha.

well, that would work OK for maintainers, but would be kind of strange=20
for folks who are pulling from such a repository.  how would that work?

my impression of git is that you don't change stuff that's already=20
committed.  you revert changes by applying a new commit that backs out=20
the original changes.  i'm speculating, but i suspect that's why there's =

a "stg pick --reverse" and not a "stg uncommit."

From: Petr Baudis
Date: Tuesday, February 14, 2006 - 3:29 pm

Dear diary, on Tue, Feb 14, 2006 at 09:58:02PM CET, I got a letter

It is ok as long as you know what are you doing - if you don't push out
the commits you've just "undid" (or work on a public accessible
repository in the first place, but I think that's kind of rare these
days; quick survey - does anyone reading these lines do that?), there's
nothing wrong on it, and it gives you nice flexibility.

For example, to import bunch of patches (I guess that's the original
intention behind this) you just run git-am on them and then stg uncommit
all of the newly added commits.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams
-

From: Sam Vilain
Date: Tuesday, February 14, 2006 - 5:22 pm

Yes, and this is one problem I envision with publishing a git repository
with an stgit stack applied - somebody later doing a pull of it will not
find the head revision they had.  I'm not sure what the net effect of
this will be, though.

Sam.
-

From: Shawn Pearce
Date: Tuesday, February 14, 2006 - 5:35 pm

It would cause some pain for anyone pulling from it with git-pull, as
git-pull won't happily go backwards from what I've seen. But I think
you can force it to do so even if it won't make sense during the
resulting merge, which then leaves the user in an interesting state.

This is actually why pg-rebase doesn't care what you move to
when you grab the remote's commit; it just jumps to that commit
and pushes your patch stack back down onto it.  So if the remote
rebuilds itself through a new commit lineage which you have never
seen before the next pg-rebase will still update to it.  But on
the other hand if you have a commit that isn't in your local patch
stack its gone into the bit bucket.

Publishing a repository with a stg (or pg) patch series isn't
a problem; the problem is that no clients currently know how to
follow along with the remote repository's patch series.  And I can't
think of a sensible behavior for doing so that isn't what git-core is
already doing today for non patch series type clients (as in don't go
backwards by popping but instead by pushing a negative delta).  :-)

-- 
Shawn.
-

From: Petr Baudis
Date: Tuesday, February 14, 2006 - 6:14 pm

Dear diary, on Wed, Feb 15, 2006 at 01:35:10AM CET, I got a letter

New Cogito will automagically do the right thing if you are just
fast-forwarding and you are using cg-update - if the branch rebased, it
will happily follow (but cg-fetch + cg-merge will NOT and it will fall
back to the tree merge).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams
-

From: J. Bruce Fields
Date: Tuesday, February 14, 2006 - 9:11 pm

If you represent each patch as a branch, with each modification to the
patch a commit on the corresponding branch, and each "push" operation a
merge from the branch corresponding to the previous patch to a branch
corresponding to the new patch (isn't that what pg's trying to do?),
then it should be possible just to track the branch corresponding to the
top patch.

In theory I guess it should also be possible to merge patch series that
have followed two lines of development, by merging each corresponding
branch.

The history would be really complicated.  You'd need to figure out how
to track the patch comments too, and you'd need scripts to convert to
just a simple series of commits for submitting upstream.  Probably not
worth the trouble, but I don't know.

If you really want revision control on patches the simplest thing might
be just to run quilt or Andrew Morton's scripts on top of a git
repository--the documentation with Andrew's scripts recommends doing
that with CVS.

--b,
-

From: Shawn Pearce
Date: Tuesday, February 14, 2006 - 11:54 pm

Yes that's pg in a nutshell.

But what happens when I pop back two patches (of three) and then push
down a different (fourth) patch?  The tree just rewound backwards
and then forwards again in a different direction.  (I apologize for
not being able to draw a nice ASCII art diagram of this, that's a
skill I'll have to learn to keep up with you guys.)  This is the
issue with Junio's pu branch in git.git and is why some people
apparently don't follow it.

StGIT and pg aren't the only ones who suffer from this wonderful

Of course.  If I delete all of the refs used by pg to mark the patch
boundaries its just another GIT branch.  Ditto for StGIT.  So clearly
you can merge them together just like any other GIT branch.

The open question is could you preserve the patch boundaries
while doing the merge.  Probably not.  It would become way to
complicated as you would want to merge the entire branch and not each
individual patch as the individual patch merges may not work but the
larger branch merge might go through without human intervention.
Of course you can try to keep the patch boundaries by exporting
all of the patches from the one branch and push them on top of
the current branch.  But isn't that what a 3 way merge is anyway?
And again that might not work as well as taking the larger patch

I think I'm almost there with pg.  One of my next tasks is the
patch log ripping code.  This is really only complicated because GIT
won't let me store the base of a 3 way merge as part of a commit;
all I can store is the set of parents.  If I had the base in the
commit (and specifically marked as such so I can tell it from the
end points) then I could easily walk through the log to extract all
commits relevant to a patch and seek forward and backward over it.

Perhaps I could cheat and record 3 parents: (HEAD, base, last).
I wonder what gitk would make of that mess.  I doubt it would display

True but you also then run into problems about needing to know which
base each patch revision ...
From: J. Bruce Fields
Date: Wednesday, February 15, 2006 - 12:45 pm

So you've got p1, p2, and p3 applied, each with its corresponding
branch--respectively, b1, b2, and b3.  Popping two patches just checks
out b1, and doesn't affect the repository at all.  If you push a new
patch, p4, you've just created a new branch, b4--you haven't touched the
existing branches.  If you push p2 and p3 back on, you're just merging
the new changes from b4 into b2 and then merging the newly merged b2
into b3.

From the point of view of someone tracking b3, this is all fine.  OK,
maybe it's excessively complicated, but pulls should work, because it
never sees history diseappear as it does when you represent each patch

Right, so you keep the tree under revision control as well as the
patches.

--b.
-

From: Junio C Hamano
Date: Thursday, February 16, 2006 - 3:24 am

I think I know what you are talking about.  Maybe you might be
interested to take a look at TO script in my todo branch [*1*]?

Also there is a sample hook script templates/hooks--pre-rebase 
that uses the same idea used in the said script.  These are what
I use to manage the "next" branch (an aggregation of topic
branches being cooked).

By the way, please do *not* do this:

    Mail-Followup-To: "J. Bruce Fields" <bfields@fieldses.org>,
            Sam Vilain <sam@vilain.net>, Petr Baudis <pasky@suse.cz>,...
	    ...

I wanted to reply to *you*, but by having the header you robbed
about 30 seconds from me, forcing me to edit the "To:"
addressee.


[Footnote]

*1* todo branch does not have *any* ancestry relationship with
my primary branches, so please do not try to merge it in your
primary repository (unless you know what you are doing).  Make a
clone into a separate directory and running "git checkout todo"
there is the cleanest way to peek into it.



-

From: Catalin Marinas
Date: Thursday, February 16, 2006 - 3:33 am

I think that's a "feature" of mutt that I couldn't understand. Every

--
Catalin
-

From: Fernando J. Pereda
Date: Thursday, February 16, 2006 - 3:42 am

That's because you told mutt you are subscribed to that list, so mutt
won't add you to Mail-Followup-To:, so you don't get duplicates. If you
don't tell mutt you are subscribed to the list, It will add your own
addres there.

I think it is a nice feature, although it seems to annoy Junio :)

Cheers,
Ferdy

--=20
Fernando J. Pereda Garcimart=EDn
Gentoo Developer (Alpha,net-mail,mutt,git)
20BB BDC3 761A 4781 E6ED  ED0B 0A48 5B0C 60BD 28D4
From: Junio C Hamano
Date: Thursday, February 16, 2006 - 3:52 am

Rightfully so.

Last I heard that was a feature mutt people regret.  Go back to
the mail archive for details -- you robbed me 30 seconds so I
won't do a research for you this time as I usually do ;-).

The "feature" is to allow you be lazy and not filter on your own
end and force everybody who wants to respond to you to fix up
the addressee header.  In other words, it is not a "feature" for
people who receives your mail at all.

-

From: Catalin Marinas
Date: Thursday, February 16, 2006 - 4:10 am

For Gnus users:

(setq message-use-mail-followup-to nil)

-- 
Catalin
-

From: Catalin Marinas
Date: Wednesday, February 15, 2006 - 10:25 am

This is a sensible way of using an uncommit command but I initially
thought it would be better to make things harder for people wanting to
re-write the history. Anyway, I'll keep this command on my todo list.

--
Catalin
-

From: Karl
Date: Thursday, February 16, 2006 - 12:54 am

stgit rewrites history all the time anyway. And as far as I recall,
there's nothing in the documentation that warns the user not to
publish stgit-managed branches. :-)

-- 
Karl Hasselstr
From: Karl
Date: Thursday, February 16, 2006 - 9:27 pm

Here is that uncommit command I was going on and on and on about.
There's also some stricter checking that refs/heads/bases is reset to
HEAD whenever we reach zero applied patches, since otherwise you can't
uncommit patches on an empty stomach.

Note the extremely cool feature that you can uncommit regardless of
how dirty your working tree is!

-- 
Karl Hasselstr
From: Karl
Date: Thursday, February 16, 2006 - 9:31 pm

Save the current HEAD into refs/heads/base if the stack is empty after
a patch has been deleted. This was not done before, which caused
refs/heads/base to not be updated after 'stg commit'. To guard against
existing repositories with no applied patches and HEAD !=
refs/heads/base, also do the update every time someone asks for the
name of refs/heads/base.

Signed-off-by: Karl Hasselstr
From: Karl
Date: Thursday, February 16, 2006 - 9:31 pm

Add an uncommit command, which is exactly the opposite of 'stg
commit'.

Signed-off-by: Karl Hasselstr
From: Catalin Marinas
Date: Sunday, February 19, 2006 - 3:51 am

Karl Hasselstr
From: Karl
Date: Sunday, February 19, 2006 - 6:45 am

On 2006-02-19 10:51:06 +0000, Catalin Marinas wrote:

> Karl Hasselstr
From: Karl
Date: Sunday, February 19, 2006 - 7:47 am

On 2006-02-19 14:45:58 +0100, Karl Hasselstr
From: Sam Vilain
Date: Sunday, February 19, 2006 - 2:15 pm

Karl Hasselstr
From: Catalin Marinas
Date: Monday, February 20, 2006 - 10:20 am

On 19/02/06, Karl Hasselstr
From: Karl
Date: Monday, February 20, 2006 - 10:30 am

On 2006-02-20 17:20:47 +0000, Catalin Marinas wrote:

> On 19/02/06, Karl Hasselstr
From: Catalin Marinas
Date: Monday, February 20, 2006 - 3:49 pm

On 20/02/06, Karl Hasselstr
From: Karl
Date: Tuesday, February 21, 2006 - 12:55 am

On 2006-02-20 22:49:22 +0000, Catalin Marinas wrote:

> On 20/02/06, Karl Hasselstr
From: Karl
Date: Wednesday, February 15, 2006 - 3:11 am

On 2006-02-14 15:58:02 -0500, Chuck Lever wrote:

> Karl Hasselstr
From: Andreas Ericsson
Date: Wednesday, February 15, 2006 - 3:42 am

Karl Hasselstr
From: Karl
Date: Wednesday, February 15, 2006 - 4:25 am

On 2006-02-15 11:42:55 +0100, Andreas Ericsson wrote:

> Karl Hasselstr
From: Karl
Date: Wednesday, February 15, 2006 - 4:27 am

On 2006-02-15 12:25:02 +0100, Karl Hasselstr
From: Catalin Marinas
Date: Friday, February 17, 2006 - 2:57 pm

I pushed tonight 2 patches for this. One of them adds a --stdout option
to 'export' so that you can see the patches. The other patch adds a
--mbox option to 'mail' that generates an mbox file on the stdout. This
is useful not only for reviewing patches.

-- 
Catalin
-

From: Sam Vilain
Date: Sunday, February 12, 2006 - 7:49 pm

How do I edit the description of an existing patch using pg?  Perhaps an
option to pg-push ?

Sam.
-

From: Shawn Pearce
Date: Sunday, February 12, 2006 - 8:29 pm

There isn't any description associated with a patch beyond its name
(which can be changed with pg-rename).  Unlike StGIT pg currently
doesn't store a description with each patch.

This is partly because I want pg to extract the comments given to
pg-ci to make the description of the patch during an export with
pg-export - but I haven't written the code to walk back along the
related commits and extract each comment.  On the other hand this
might not be the best description for a patch.  :-)

-- 
Shawn.
-

From: Sam Vilain
Date: Sunday, February 12, 2006 - 9:40 pm

ok.  Well, perhaps a nice solution might be just to aggregate the
comments as each new commit is made.  ie, the previous comment is
prepended to the new comment unless you use the editor or a special
-M (or whatever) option that replaces the running comment.

I tried importing a patchset into pg, and made some changes to it to see
the patch revisioning going on.  However, I can't see this happening.
Can you perhaps include this information in your tutorial?

As far as other, more general critiques of the software goes:  What
about merging?  stgit has a very nice way of merging; I specify how to
merge using a config file, and when I rebase my patches with "stg pull",
it fires up my custom editor.  All I really want is a way to specify how
to handle merges, with the ancestor/left/right files on hand.  I want to
use something as simple as this script:

#!/bin/sh

branch1="$1"
branch2="$2"
ancestor="$3"
output="$4"

echo "Merging:"
echo
echo "   $branch1"
echo " - $ancestor"
echo
echo " with:"
echo
echo "   $branch2"
echo " - $ancestor"
echo
echo " to: $output"
echo ""
echo -n "Trying diff3..."

if diff3 -L local -L older -L remote -m -E "$branch1" "$ancestor" \
    "$branch2" > "$output"
then
     echo "OK"
else
     echo "failed"
     echo "falling back to ediff-merge"
     emacs --eval "(ediff-merge-files-with-ancestor \"${branch1}\"
                    \"${branch2}\" \"${ancestor}\" nil \"${output}\")"
fi

Those commands I got from the default .stgitrc config.

That's all the features I'm really after.

Sam.
-

From: Shawn Pearce
Date: Sunday, February 12, 2006 - 11:03 pm

Yea, that's not a bad idea.  If you are creating a new commit you
probably would want to edit the running description for the patch;

Revisioning doesn't happen for the series, just the individual
patches.  But I've thought about series revisoning and keeping a
secondary GIT index/commit chain external to the main repository
for exactly this purpose.

Each change to a patch (pg-ci) is a new commit object in GIT with
the prior commit object as its parent; if you use pg-ci a few times
with the same patch on the stack then look at the log with git-log
or gitk you'll see the commits are chained together.

When you pop patches and reorder them in the series the resulting
merges are stored as commits with two parents: one for the HEAD
at the time of the merge and one for the commit which was the last
commit in the patch being pushed (HEAD^1 and HEAD^2 respectively).
For example:

	pg-new A
	echo a >>somefile
	pg-ci -m"This is a"
	pg-new B
	echo b >>somefile
	pg-ci -m"This is b"

	pg-pop -a
	pg-push B  # base used to be HEAD+A, now its HEAD
	pg-push A  # base used to be HEAD, now its HEAD+B

The challenge then becomes walking through the merge history.
If you look at pg's own history you'll see an interesting knot
in gitk at a7e73545e511c5c2daea1f6c7bf06cf3179e7f0da (Refreshed
patch Create-Rebase-Tool).  This was produced because I reorded
the patches in the stack and thus had to merge them.  It was an
automatic merge, but it still generated merge commit objects.

Good suggestion about including some details about it in the

pg doesn't currently invoke any user code when an automatic merge
fails during pg-push or pg-rebase.  It does attempt to produce
a 3 way merge and leaves the resulting portions for you in the
filesystem.  If you look at MERGING.txt you'll see that up to 5
files can come out of a merge (here I'm using the tracked file X.c):

	X.c
	X.c-head
	X.c-last
	X.c-pbase
	X.c-rej

These just get left in the filesystem for you to use as you want;
in your ...
From: Catalin Marinas
Date: Monday, February 13, 2006 - 7:40 am

Couldn't help replying to such a topic :-) (only that the ":" ending
of the above phrase might make people think that some features you
listed are not available in StGIT).

Without much testing, I think pg is a good tool but it is different
from StGIT in many ways. It mainly resembles the topic branches way of
working with the advantage of having them stacked on each-other. Each
patch seems to be equivalent to a topic branch where you can commit
changes. Rebasing a patch is equivalent to a merge in a branch with
the merge commit having a general description like "Refreshed patch
..." and two parents - the new base and the old top.

While I don't say the above is a bad thing, it is pretty different
from StGIT. With StGIT, the history of the tree only shows one commit
per patch with the patch description chosen by the user. If you edit
the description or modify the patch, the old patch or description is
dropped from the main branch (visible via HEAD) and you only get the
latest one. This clean history has many advantages when sending

This is the case with StGIT as well. It doesn't usually require the

There have been discussions to adding this to StGIT as well (and there
is a patch already from Chuck). It is a good thing to have but I'm
opposed to the idea of having the history accessible from the top of
the patch. Since the patch can be refreshed indefinitely, it would
make the main history (visible from HEAD) really ugly and also cause
problems with people pulling from a tree. I prefer to have a separate

That's missing indeed in StGIT but it will be available in the next
release. I didn't push this yet because I wasn't sure what to do with

StGIT has been doing this from the beginning. You would need to run a
'stg clean' after a rebase (or push). I prefer to run this command
manually so that 'stg series -e' would show the empty patches and let

Might be possible but I haven't done any tests. There are some
optimisations in StGIT that make it pretty fast: (1) if the base ...
From: Shawn Pearce
Date: Monday, February 13, 2006 - 9:56 pm

Yes.  I didn't intend on exporting the entire patch history for
delivery upstream; I only intended on exporting the batch between
its base and last markers, which amounts to giving a single diff
such as what StGIT would generate.  But I had planned on pulling
the commit comments from all history into the header of the patch

I definately agree.  I have been rather unhappy with the log
structure that pg is giving me when I flip patches around on the
stack.  So I'm certainly considering keeping the history of the
patch in a parallel tree stored within the same object and refs

I see you actually already pushed out a change for this for StGIT.
That's good news.  :-) I noticed the solution StGIT used is close to
pg's, except that StGIT has the simplified single-commit-per-patch

Actually StGIT didn't do this correctly for one of my use cases
and that's one of the things that drove me to trying to write pg
(because I wondered if there was a way to resolve it automatically).
Try building a patch series such as:

	... start with an empty stack ...

	... create patch A ...
	... edit file hello.c ...
	... refresh patch A ...

	... create patch B ...
	... edit file hello.c (same line region as patch A) ...
	... refresh patch B ...

	... generate patch A+B (as one patch!) ...
	... send A+B upstream ...

	... pull upstream down ...

StGIT seemed to not handle this when it tried to reapply the two
already applied patches.  A won't apply because the file coming
down is actually A+B, not A's predecessor and not A.  B won't apply
because the file also isn't A (B's predecessor).

pg resolves this by attempting to automatically fold patches during
a pg-rebase (equiv. of stg pull).  If a patch fails to push cleanly
and there's another patch immediately behind it which also should
be reapplied pg aborts and retries pushing the combination of the
patches.  This fixes my A+B case quite nicely during a rebase.  :-)

Of course it doesn't deal with the upstream giving me A+B+C and ...
From: Shawn Pearce
Date: Monday, February 13, 2006 - 11:14 pm

I did a quick benchmark of the latest StGIT and pg versions available
by cloning a linux repository twice (pg and stg) and constructing
an identical patch in in each which modified the same set of files
(3 files in 3 different subdirectories).

Popping and pushing this patch of 3 files is a fast forward/rewind
case to both implementations, but pg was faster for me:

stg pop; stg push (fast-forward)
  pop  real 0m13.824s user 0m1.904s sys 0m3.934s
  pop  real 0m13.670s user 0m1.882s sys 0m3.973s
  pop  real 0m21.070s user 0m2.005s sys 0m4.069s
  pop  real 0m15.346s user 0m1.757s sys 0m3.846s
  pop  real 0m16.960s user 0m1.888s sys 0m3.866s
          
  push real 0m20.650s user 0m2.027s sys 0m4.015s
  push real 0m15.624s user 0m1.958s sys 0m3.966s
  push real 0m13.277s user 0m1.746s sys 0m3.796s
  push real 0m12.739s user 0m1.764s sys 0m3.822s
  push real 0m15.161s user 0m1.973s sys 0m3.939s
        
pg-pop; pg-push (fast-forward)
  pop  real 0m10.009s user 0m1.919s sys 0m2.265s
  pop  real 0m 4.710s user 0m1.692s sys 0m1.560s
  pop  real 0m 4.333s user 0m1.664s sys 0m1.554s
  pop  real 0m 5.480s user 0m1.848s sys 0m1.638s
  pop  real 0m 4.412s user 0m1.680s sys 0m1.604s
          
  push real 0m5.813s user 0m1.750s sys 0m1.733s
  push real 0m4.345s user 0m1.686s sys 0m1.632s
  push real 0m5.326s user 0m1.721s sys 0m1.658s
  push real 0m4.740s user 0m1.691s sys 0m1.647s
  push real 0m4.487s user 0m1.702s sys 0m1.637s


I tried to do the merge case which requires reconstructing the
patch onto a new base revision.  This was easy to test over and over
again on pg (pg-rebase; pg-rebase --undo) but I don't know how I can
safely undo the stg pull so I can repeat it on the stg repository.
Instead I tested git-diff-tree|git-apply as that's what StGIT uses
internally.  StGIT does a check for a clean tree before starting
its merge so I cheated here and used the pg version of that check
as part of the diff/apply cost to try and make it slightly more
fair to ...
From: Catalin Marinas
Date: Wednesday, February 15, 2006 - 10:20 am

For the simple tests you did the difference is not that big. It
becomes a real problem when there are many file deletions/additions in
the upstream tree since git-read-tree doesn't handle them and
git-merge-index would need to call the external tool for each of them.

To test the above, clone the 2.6.12 kernel version, create some
trivial patches and rebase to 2.6.16-rc3. StGIT was running even for 5
minutes per patch before implementing the diff-tree/apply method.

--
Catalin
-

From: Catalin Marinas
Date: Wednesday, February 15, 2006 - 10:12 am

You are right, if two patches modify the same line and both were
merged upstream, the three-way merging would report a conflict for the
first patch and maybe the second (depending on how the first conflict

But what would happen if there was a third-party patch that's
modifying the same line? A+B application would fail in this case. Does
pg go back to only apply A and report a conflict?

There is another problem with this approach if you have tens of
patches. Would pg try to fold all of them?

Some time ago I had a look at Darcs and its patch theory (patch
commuting). Their approach to conflicts was to include the conflicts
in patch A and propagate them to the last patch to be merged. It's
like creating two versions of the conflicting hunk, one of them
corresponding to the local tree (that in patch A) and the other to the
upstream tree. Merging patch B is only done in the local hunk in the
end both conflicting hunks would be identical and one of them removed.

While the above algrithm seems to work OK in Darcs (but quite resource
intensive), it's pretty hard to implement and I don't think it's worth
for a small number of cases this could occur.

--
Catalin
-

From: Shawn Pearce
Date: Wednesday, February 15, 2006 - 10:55 am

When this occurs pg just gives up and leaves both patches A and
B unapplied and gives you the list of patches which it couldn't
apply but wanted to.  The working directory is left clean; its the
new base plus whatever patches before A that did apply cleanly.
I could have pg go back and try pushing A again and leave the
conflict ready for you to resolve but I don't always want that.
Since the user can have that happen with a quick no-arg `pg-push`
I leave it to the user to retry pushing A if they really think
that's worth trying.

However if the last patch fails to push during a pg-rebase then pg
leaves it alone and your working directory is dirty and you are left
with that last patch partially applied.  At which point you can back

Yea.  Which might not be pretty.  10 patches would cause pg to
attempt applying 11 patches before giving up, but each time the patch
is increased in size to include its predecessors who also didn't
apply cleanly.  As soon as a larger cluster applies pg goes back to
trying single patch application.  Obviously this could take a while
as the patch size is growing on each attempt and we are duplicating
work every time as pg always starts from a clean working directory.

Example: Say I have A, B, C, D, E, F on the stack.  A wasn't provided
by the upstream and pushes down cleanly.  B+C+D was given to me
by the upstream so pg first tries B, fails, then B+C, fails, then
B+C+D, succeeds, so it folds B+C+D into D and finishes pushing D.
Then it tries E, if E succeeds it tries F on its own.  If E fails it
tries E+F.  What's left in the working directory depends on if the
last operation was an auto-fold attempt or not and if it applied

Hmm.  I had looked at Darcs over a year ago and found it to be a
rather interesting idea but at the time it couldn't handle my ~7000
file tree (and GIT wasn't even getting started yet).  I was actually
thinking about trying to drag the rejecting hunks forward somehow
when doing the auto-folding but I hadn't quite found a way ...
Previous thread: [PATCH] delta micro optimization by Nicolas Pitre on Friday, February 10, 2006 - 11:42 am. (3 messages)

Next thread: [PATCH] Ignore commits for which cvsps can't identify a branch by Christian Biesinger on Friday, February 10, 2006 - 2:02 pm. (6 messages)