Re: git sequencer prototype

Previous thread: [PATCH/RFC] Emulating symlinks on Windows with file copies by Patrick Higgins on Monday, June 30, 2008 - 4:54 pm. (1 message)

Next thread: none
From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:38 pm

Hi,

here is the patchset for the git-sequencer prototype, documentation,
test suite and a first git-am and git-rebase-i migration.
Indeed, monster patches. ;)

I'm using sequencer-based git-am and git-rebase-i and also git-sequencer
itself for around 2-3 weeks now. So, for me, it is reality-proven, but
I'm curious about your opinions/suggestions in usage and source.

The migration patches are a little hard to code-review in the diff-form,
but feel free to apply, test, and then look at the code ;)

Regards,
  Stephan

--

From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:38 pm

git sequencer is planned as a backend for user scripts
that execute a sequence of git instructions and perhaps
need manual intervention, for example git-rebase or git-am.

Mentored-by: Christian Couder <chriscool@tuxfamily.org>
Mentored-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
Some comments:
 - It's declared as plumbing, but the prototype can't really be
   named "plumbing", imho, since it uses porcelain like git-commit.
 - I hope it is obvious that parts of it are based on
   git-rebase-i and git-am code, so if you stumble over code
   that looks familiar, it is not necessarily by accident.

 .gitignore       |    1 +
 Makefile         |    1 +
 command-list.txt |    1 +
 git-sequencer.sh | 1902 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 1905 insertions(+), 0 deletions(-)
 create mode 100755 git-sequencer.sh

diff --git a/.gitignore b/.gitignore
index 4ff2fec..65a075b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -111,6 +111,7 @@ git-revert
 git-rm
 git-send-email
 git-send-pack
+git-sequencer
 git-sh-setup
 git-shell
 git-shortlog
diff --git a/Makefile b/Makefile
index bf77292..db93956 100644
--- a/Makefile
+++ b/Makefile
@@ -250,6 +250,7 @@ SCRIPT_SH += git-rebase--interactive.sh
 SCRIPT_SH += git-rebase.sh
 SCRIPT_SH += git-repack.sh
 SCRIPT_SH += git-request-pull.sh
+SCRIPT_SH += git-sequencer.sh
 SCRIPT_SH += git-sh-setup.sh
 SCRIPT_SH += git-stash.sh
 SCRIPT_SH += git-submodule.sh
diff --git a/command-list.txt b/command-list.txt
index 3583a33..44bb5b0 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -101,6 +101,7 @@ git-rev-parse                           ancillaryinterrogators
 git-rm                                  mainporcelain common
 git-send-email                          foreignscminterface
 git-send-pack                           synchingrepositories
+git-sequencer                           plumbingmanipulators
 git-shell                               ...
From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:38 pm

Mentored-by: Christian Couder <chriscool@tuxfamily.org>
Mentored-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
 Documentation/git-sequencer.txt |  348 +++++++++++++++++++++++++++++++++++++++
 1 files changed, 348 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/git-sequencer.txt

diff --git a/Documentation/git-sequencer.txt b/Documentation/git-sequencer.txt
new file mode 100644
index 0000000..e0c6410
--- /dev/null
+++ b/Documentation/git-sequencer.txt
@@ -0,0 +1,348 @@
+git-sequencer(1)
+================
+
+NAME
+----
+git-sequencer - Execute a sequence of git instructions
+
+SYNOPSIS
+--------
+[verse]
+'git-sequencer' [--batch] [--onto=<base>] [--verbose|--quiet] [<file>]
+'git-sequencer' --continue | --skip | --abort | --edit | --status
+
+
+DESCRIPTION
+-----------
+Executes a sequence of git instructions to HEAD or `<base>`.
+The sequence is given by `<file>` or standard input.
+Also see 'TODO FILE FORMAT' below.
+
+Before doing anything, the TODO file is checked for correct syntax
+and sanity.
+
+In case of a conflict or request in the TODO file, git-sequencer will
+pause. On conflict you can use git-diff to locate the markers (`<<<<<<<`)
+and make edits to resolve the conflict.
+
+For each file you edit, you need to tell git the changes by doing
+
+    git add <file>
+
+After resolving the conflict manually and updating the index with the
+desired resolution, you can continue the sequencing process with
+
+    git sequencer --continue
+
+Alternatively, you can undo the git-sequencer progress with
+
+    git sequencer --abort
+
+or skip the current instruction with
+
+    git sequencer --skip
+
+or correct the TODO file with
+
+    git sequencer --edit
+
+During pauses or when finished with the sequencing task, the current
+HEAD will always be the result of the last processed instruction.
+
+
+OPTIONS
+-------
+<file>::
+	Filename of the TODO file.  If omitted, standard ...
From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:38 pm

Mentored-by: Christian Couder <chriscool@tuxfamily.org>
Mentored-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
Note that the --quiet tests are test_expect_failure, because I
have not really cared about the output behavior of sequencer,
and --quiet cannot be a trivial exec >/dev/null, because sometimes
editors are invoked, etc.

Also note that the fake-editor is quite different from the one
in the rebase tests.  According to a "session", the file is
totally rewritten. It's not possible to do something like
FAKE_LINES="3 1 2" so that the third line is the first one, etc.

 t/t3350-sequencer.sh |  810 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 810 insertions(+), 0 deletions(-)
 create mode 100755 t/t3350-sequencer.sh

diff --git a/t/t3350-sequencer.sh b/t/t3350-sequencer.sh
new file mode 100755
index 0000000..33522eb
--- /dev/null
+++ b/t/t3350-sequencer.sh
@@ -0,0 +1,810 @@
+#!/bin/sh
+#
+# Copyright (c) 2008 Stephan Beyer
+#
+# `setup' is based on t3404* by Johannes Schindelin.
+
+test_description='git sequencer
+
+These are basic usage tests for git sequencer.
+'
+. ./test-lib.sh
+
+# set up two branches like this:
+#
+# A - B - C - D - E
+#   \
+#     F - G - H
+#       \
+#         I
+#
+# where B, D and G touch increment value in file1.
+# The others generate empty file[23456].
+
+SEQDIR=".git/sequencer"
+SEQMARK="refs/sequencer-marks"
+MARKDIR=".git/$SEQMARK"
+
+test_expect_success 'setup' '
+	: >file1 &&
+	git add file1 &&
+	test_tick &&
+	git commit -m "generate empty file1" &&
+	git tag A &&
+	echo 1 >file1 &&
+	test_tick &&
+	git commit -m "write 1 into file1" file1 &&
+	git tag B &&
+	: >file2 &&
+	git add file2 &&
+	test_tick &&
+	git commit -m "generate empty file2" &&
+	git tag C &&
+	echo 2 >file1 &&
+	test_tick &&
+	git commit -m "write 2 into file1" file1 &&
+	git tag D &&
+	: >file3 &&
+	git add file3 &&
+	test_tick &&
+	git commit -m "generate ...
From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:38 pm

This patch also adds --abort to git-am, which is just a
trivial implication of using git-sequencer.

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
Hm, what to say here?

I have to admit that interactive mode is purely tested.
Well, I've used sequencer-based git-am for a while, but never really
used the interactive mode (except for first manual test cases).

 Documentation/git-am.txt |    5 +-
 git-am.sh                |  552 +++++++++++++---------------------------------
 git-rebase.sh            |    7 +-
 t/t3407-rebase-abort.sh  |   10 +-
 t/t4150-am.sh            |   21 ++-
 5 files changed, 178 insertions(+), 417 deletions(-)

diff --git a/Documentation/git-am.txt b/Documentation/git-am.txt
index 46544a0..b714106 100644
--- a/Documentation/git-am.txt
+++ b/Documentation/git-am.txt
@@ -13,7 +13,7 @@ SYNOPSIS
          [--3way] [--interactive] [--binary]
          [--whitespace=<option>] [-C<n>] [-p<n>]
          <mbox>|<Maildir>...
-'git-am' [--skip | --resolved]
+'git-am' (--abort | --skip | --resolved)
 
 DESCRIPTION
 -----------
@@ -79,6 +79,9 @@ default.   You could use `--no-utf8` to override this.
 --interactive::
 	Run interactively.
 
+--abort::
+	Abort applying and rewind applied patches.
+
 --skip::
 	Skip the current patch.  This is only meaningful when
 	restarting an aborted patch.
diff --git a/git-am.sh b/git-am.sh
index 2c517ed..dd19dd7 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -6,9 +6,11 @@ SUBDIRECTORY_OK=Yes
 OPTIONS_KEEPDASHDASH=
 OPTIONS_SPEC="\
 git-am [options] <mbox>|<Maildir>...
+git-am [options] --abort
 git-am [options] --resolved
 git-am [options] --skip
 --
+abort           abort patching and reset done patches
 d,dotest=       (removed -- do not use)
 i,interactive   run interactively
 b,binary        pass --allow-binary-replacement to git-apply
@@ -30,104 +32,101 @@ set_reflog_action am
 require_work_tree
 cd_to_toplevel
 
-git var GIT_COMMITTER_IDENT >/dev/null || exit
+git var GIT_COMMITTER_IDENT >/dev/null ...
From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:39 pm

Hi,

the patches for rebase-i migration follow.

Why this intermediate mail?
Because the patches will not apply to master/next/pu/whatever.
It needs js/rebase-i-sequencer, that is "dropped for now" from
git.git.  So you have to do some extra work to apply.

Well, I could've just added a "Squashed js/rebase-i-sequencer patch"
or something, but
 1. I didn't write the patches so I don't want to claim I'm the author,
 2. It's an opportunity to try sequencer, as follows:

If you still have the commits (a freshly cloned repo hasn't),
you can do:

git sequencer <<EOF
#pick 367bcc9cbc3d903c076f # optional: "fake-editor: output TODO list if unchanged"
pick 0fa413ebc6e073a52d88
pick -R 88b1f0b8094638b1f953 # just to prevent a silly conflict
pick 966fdf0ab272a64250fb
pick 45d6022d611fd86a13c5
pick 051e0cfb88cd1d2a24bd
# note: the following patches will make t3404 fail until migration to sequencer
pick d9711a80796fd00169ca
pick 10046a6f8adaccac0ea1
pick abe32eafcd8302c387ed
pick d481bcc93f393bbfaaef
pick dbcd70439f697deeb317
pick b1ba88d812434a51889f
pick cc76fdac36ff25eaa77b
pick f8e037d76a224bd87236
pick 777790a3bdfeb91ee4a6
pick 7cdf70106412f44b9116
pick 14a4f1e0a804ccc905d7
EOF

If this results in a lot of "is not a commit" errors, then do
	git sequencer --abort
	(You may also have a look at git sequencer --status, if you like)
and fetch the missing commits:
	git fetch git://repo.or.cz/git/sbeyer.git seq-proto-dev2
and then redo the example above.

When the patches are applied, the following patches will apply, too.

Regards,
  Stephan

PS: Sorry, for the nastiness of the patch application ;-)

--

From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:39 pm

Also add some checks that --continue/--abort/--skip
actions are used without --onto, -p, -t, etc.

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
 git-rebase--interactive.sh |   72 ++++++++++++++++++++++++++++---------------
 1 files changed, 47 insertions(+), 25 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7e073bb..ad16fa2 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -10,10 +10,27 @@
 # The original idea comes from Eric W. Biederman, in
 # http://article.gmane.org/gmane.comp.version-control.git/22407
 
-USAGE='(--continue | --abort | --skip | [--preserve-merges] [--first-parent]
-	[--preserve-tags] [--verbose] [--onto <branch>] <upstream> [<branch>])'
+OPTIONS_KEEPDASHDASH=
+OPTIONS_SPEC="\
+git-rebase [-i] [options] [--] <upstream> [<branch>]
+git-rebase [-i] (--continue | --abort | --skip)
+--
+ Available options are
+p,preserve-merges  try to recreate merges instead of ignoring them
+t,preserve-tags    update tags to the new commit object
+m,merge            always used (no-op)
+i,interactive      always used (no-op)
+onto=              rebase onto given branch instead of upstream
+v,verbose          display a diffstat of what changed upstream
+ When preserving merges
+f,first-parent     show only commits following the first parent of each commit
+s,strategy=        use the given merge strategy
+ Actions:
+continue           continue rebasing process
+abort              abort rebasing process and restore original branch
+skip               skip current patch and continue rebasing process
+"
 
-OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
 
@@ -25,6 +42,8 @@ SQUASH_MSG="$DOTEST"/message-squash
 PRESERVE_MERGES=
 STRATEGY=
 VERBOSE=
+ONTO=
+MARK_PREFIX='refs/rebase-marks'
 test -f "$DOTEST"/strategy && STRATEGY="$(cat "$DOTEST"/strategy)"
 test -f "$DOTEST"/verbose && VERBOSE=t
 
@@ -515,10 +534,19 @@ create_extended_todo_list () {
 	done
 }
 
+is_standalone () {
+	test ...
From: Stephan Beyer
Date: Monday, June 30, 2008 - 7:39 pm

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
 git-rebase--interactive.sh    |  423 ++++++-----------------------------------
 t/t3404-rebase-interactive.sh |   29 ++--
 2 files changed, 71 insertions(+), 381 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index ad16fa2..60f43b2 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -35,28 +35,13 @@ skip               skip current patch and continue rebasing process
 require_work_tree
 
 DOTEST="$GIT_DIR/.dotest-merge"
-TODO="$DOTEST"/git-rebase-todo
-DONE="$DOTEST"/done
-MSG="$DOTEST"/message
-SQUASH_MSG="$DOTEST"/message-squash
+TODO="$DOTEST/git-rebase-todo"
 PRESERVE_MERGES=
 STRATEGY=
 VERBOSE=
+INTERACTIVE=
 ONTO=
-MARK_PREFIX='refs/rebase-marks'
-test -f "$DOTEST"/strategy && STRATEGY="$(cat "$DOTEST"/strategy)"
-test -f "$DOTEST"/verbose && VERBOSE=t
-
-GIT_CHERRY_PICK_HELP="  After resolving the conflicts,
-mark the corrected paths with 'git add <paths>', and
-run 'git rebase --continue'"
-export GIT_CHERRY_PICK_HELP
-
-mark_prefix=refs/rebase-marks/
-
-warn () {
-	echo "$*" >&2
-}
+test -f "$DOTEST"/verbose && VERBOSE=--verbose
 
 output () {
 	case "$VERBOSE" in
@@ -83,51 +68,11 @@ require_clean_work_tree () {
 
 ORIG_REFLOG_ACTION="$GIT_REFLOG_ACTION"
 
-comment_for_reflog () {
-	case "$ORIG_REFLOG_ACTION" in
-	''|rebase*)
-		GIT_REFLOG_ACTION="rebase -i ($1)"
-		export GIT_REFLOG_ACTION
-		;;
-	esac
-}
-
-last_count=
-mark_action_done () {
-	sed -e 1q < "$TODO" >> "$DONE"
-	sed -e 1d < "$TODO" >> "$TODO".new
-	mv -f "$TODO".new "$TODO"
-	count=$(grep -c '^[^#]' < "$DONE")
-	total=$(($count+$(grep -c '^[^#]' < "$TODO")))
-	if test "$last_count" != "$count"
-	then
-		last_count=$count
-		printf "Rebasing (%d/%d)\r" $count $total
-		test -z "$VERBOSE" || echo
-	fi
-}
-
-make_patch () {
-	parent_sha1=$(git rev-parse --verify "$1"^) ||
-		die "Cannot get patch for $1^"
-	git diff-tree -p "$parent_sha1".."$1" > ...
From: Stephan Beyer
Date: Saturday, July 5, 2008 - 10:31 am

Hi,

I've just noticed that those tag names:
  +	git tag A &&
  +	git tag B &&
  +	git tag C &&
  +	git tag D &&
  +	git tag E &&
  +	git tag F &&
  +	git tag I &&
  +ref refs/tags/CE
are perhaps a bad idea, because they could also be an abbreviated SHA1.
As it seems, git does not think this is ambiguous, since the exact tag
name exists.
So it could make sense to add a non-[0-9A-Fa-f] prefix, but it does not
seem to be necessary.  Hmmm, so this mail is just a mental note :)

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F
--

From: Junio C Hamano
Date: Saturday, July 5, 2008 - 11:16 am

I do not think we take a single letter abbrev so it should be fine.
--

From: Jakub Narebski
Date: Tuesday, July 1, 2008 - 6:02 am

I hope that it could be also used by git-cherry-pick and git-revert,

Does it run pre-commit hooks, for example checking for merge markers


Does it mean that editor will be invoked with only _unprocessed_
part of the TODO file?  It looks like it, but it might be not

Nice.





Do all options make sense?  What about `--index' and `--cached',



How do you squash merges?  Creating merge with an union of parents,
excluding commits which got squashed?

It means

  ...a---b---c---d         ...[abcd]
            /        ==>        /
      ...x-/               ..x-/

but

  ...a---b---c---d          ...[abcd]
      \     /        ==>

Don't you enumerate those return values?

-- 
Jakub Narebski
Poland
ShadeHawk on #git
--

From: Stephan Beyer
Date: Tuesday, July 1, 2008 - 9:03 am

Hi,

On Tue, Jul 01, 2008 at 06:02:54AM -0700,

Right, you've already mentioned it several times. ;-)
But I haven't included it currently, because git-sequencer currently *uses*
cherry-pick/revert to pick/revert commits, and I think this can lead to
some confusion.
And, I think there also has to be some discussion, e.g.
if even the single cherry-pick/revert uses sequencer, then the behavior
changes for the user: currently you have to "git commit" after a
conflict on cherry-pick. When it uses sequencer, you have to
git-sequencer --continue (or --skip or --abort) ;)

So I'm only concentrating on rebase and am users until the builtin sequencer

It only commits (git commit) staged changes (if the working tree is

Agreed.

diff --git a/Documentation/git-sequencer.txt b/Documentation/git-sequencer.txt
index e0c6410..1a9eda4 100644
--- a/Documentation/git-sequencer.txt
+++ b/Documentation/git-sequencer.txt
@@ -62,7 +62,7 @@ OPTIONS
 	(e.g. a conflict or the need to run an editor), git-sequencer fails.
 +
 Note that the sanity check fails, if you use this option
-and an instruction like `edit` or `pause`.
+and an instruction like `edit` or `pause` is in the TODO file.
 
 --onto=<base>::
 	Checkout given commit or branch before sequencing.

Thanks, this sounds better.

@@ -83,7 +83,7 @@ and an instruction like `edit` or `pause`.
 	operations can be done to change that status.
 
 --edit::
-	Invoke editor to edit the undone rest of the TODO file.
+	Invoke editor to edit the unprocessed part of the TODO file.
 +
 The file is syntax- and sanity-checked afterwards, so that you can
 safely run `git sequencer --skip` or `--continue` after editing.

Right. This is used throughout the document...
I thought it is clear and better to use than always describing around it
by "last created commit".

Hm, is "by" the right preposition?

@@ -150,7 +150,7 @@ The set marks are removed after the sequencer has completed.
 merge [options] <commit-ish1> <commit-ish2> ... <commit-ishN>::
 ...
From: Jakub Narebski
Date: Tuesday, July 1, 2008 - 11:04 am

Hi,




Ah.  I'm sorry.  So it is like "git branch <ref>", isn't it?

 
Ah, so squash --from <mark> picks up everything since "mark <mark>",
but does not include marked commit!  Clever!  In this case allowing

O.K.  I guess that is something left for later, especially that
forbidding merges in squashed commit is default (and you can always

Perhaps stating that it returns 128 on fatal error, or rewording
that any other return value means some fatal error (does it?)
would be better.  But that are details...

-- 
Jakub Narebski
Poland
--

From: Stephan Beyer
Date: Tuesday, July 1, 2008 - 12:50 pm

Hi,


I don't know.  If something like "last created commit" is better,


Good question. The reflog is another mistery to me that I haven't really
cared about, because I haven't used it yet myself.
At least the reflog test cases for git rebase -i in the test suite pass.


Forbidding merges is default currently. It's just a sanity check. And
the option bypasses this check.

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F
--

From: Jakub Narebski
Date: Tuesday, July 1, 2008 - 5:39 pm

Hi!


Actually I meant "git branch -f <ref>" here.  But I forgot that it


You have 'branch' (<onto>) reflog, HEAD reflog, and reflog for '<ref>'.
While 'branch' reflog should I think record only start and end of
sequencer, or rather git-am or git-rebase, I'm not sure what to do about
HEAD reflog...  We should fill reflog for <ref>, to be able to revert

Although I guess having example would make it clear from the go...

Good work!
-- 
Jakub Narebski
Poland
--

From: Junio C Hamano
Date: Tuesday, July 1, 2008 - 6:20 pm

A simple answer and a more elaborate one.

 * It is the final step of "git rebase" which detaches HEAD while it
   operates these days.

 * You can drive sequencer backend from a front-end that rewrite history

That is not very important question, as reflog updates would happen as
long as you use update-ref automatically.

Much more important question you did not ask is how it would interact with
"sequencer --abort".  Ideally it should rewind the ref update (and without
relying on the user having reflog on that ref).

I however personally feel that this "ref" thing is being a bit too
ambitious.
--

From: Stephan Beyer
Date: Tuesday, July 1, 2008 - 8:01 pm

I think if I had not started on top js/rebase-i-sequencer, I would have
not necessarily added "ref" (or "tag", as it was before).
But js/rebase-i-sequencer introduced a nice feature to
git-rebase-i:
 -t,--preserve-tags ... "update tags to the new commit object"

I think this is worth the feature ;-)


Ha ;)
That's a quite obvious truth that I have not implemented.
Btw: sequencer does the detaching/attaching when using "--onto <base>"
and <base> is a branch.
IIRC there was a reason that I also kept the detaching/attaching of
git-rebase-i itself but I can't remember. (I think I have to look


Yes, you asked the question in the RFC thread about the spec.

"Rewind the ref update" means:
 - delete new generated refs
 - update overwritten refs to their old commits
So if I keep a list containing <ref> [<oldcommit>], that behavior
is easily reached. But this is not suited for the shell prototype ;)


Btw, also the --skip case can be arguable:

	pick fa1afe1
	pick cedefe1	# conflict, will be "--skip"ped
	ref refs/tags/v1.6.2

Here the question can be, if the "ref" should be skipped, too.
But I think the just-don't-care-strategy is a good one, so

I agree that it is no must-have feature but a nice-to-have feature ;)
Especially for the --preserve-tags feature.



Yes, I see that the EXAMPLES section is really important and the
squash --from is important therein.

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F
--

From: Stephan Beyer
Date: Saturday, July 5, 2008 - 10:00 am

Mentored-by: Christian Couder <chriscool@tuxfamily.org>
Mentored-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
This contains some feedback of Jakub.

 Documentation/git-sequencer.txt |  387 +++++++++++++++++++++++++++++++++++++++
 1 files changed, 387 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/git-sequencer.txt

diff --git a/Documentation/git-sequencer.txt b/Documentation/git-sequencer.txt
new file mode 100644
index 0000000..1a9eda4
--- /dev/null
+++ b/Documentation/git-sequencer.txt
@@ -0,0 +1,387 @@
+git-sequencer(1)
+================
+
+NAME
+----
+git-sequencer - Execute a sequence of git instructions
+
+SYNOPSIS
+--------
+[verse]
+'git-sequencer' [--batch] [--onto=<base>] [--verbose|--quiet] [<file>]
+'git-sequencer' --continue | --skip | --abort | --edit | --status
+
+
+DESCRIPTION
+-----------
+Executes a sequence of git instructions to HEAD or `<base>`.
+The sequence is given by `<file>` or standard input.
+Also see 'TODO FILE FORMAT' below.
+
+Before doing anything, the TODO file is checked for correct syntax
+and sanity.
+
+In case of a conflict or request in the TODO file, git-sequencer will
+pause. On conflict you can use git-diff to locate the markers (`<<<<<<<`)
+and make edits to resolve the conflict.
+
+For each file you edit, you need to tell git the changes by doing
+
+    git add <file>
+
+After resolving the conflict manually and updating the index with the
+desired resolution, you can continue the sequencing process with
+
+    git sequencer --continue
+
+Alternatively, you can undo the git-sequencer progress with
+
+    git sequencer --abort
+
+or skip the current instruction with
+
+    git sequencer --skip
+
+or correct the TODO file with
+
+    git sequencer --edit
+
+During pauses or when finished with the sequencing task, the current
+HEAD will always be the result of the last processed instruction.
+
+
+OPTIONS
+-------
+<file>::
+	Filename ...
From: Jakub Narebski
Date: Tuesday, July 8, 2008 - 3:37 am

A few comments and ideas:

1. Splitting a patch

I cannot comment well on git-sequencer, as I have started using
StGIT patch management interface instead of git-rebase in times when
there were no "git rebase --interactive".  Nevertheless working with
StGIT is a bit similar to working with interactive rebase...

I don't find myself wanting to join two patches into one (to squadh
a commit) perhaps because when I want to add something to a commit
(to a patch) I simply go to this patch, edit files, and refresh the
patch.  From time to time however I find myself SPLITTING a patch,
for example extracting something added "by the way"/"while at it"
into separate commit (like late separate better documenting
project_index file format from adding optional description field
to project_index file format).

Currently I don't see easy way to do this with git-sequencer.


2. Patch based rebase

git-rebase by default, and for speed, uses git-format-patch / git-am
pipeline (utilizing '--rebasing' option to git-am to avoid changing
commit messages, even if they do not follow commit message conventions).
If you want for plain "git rebase" to use git-sequencer, it should be
easy to support this "engine"; therefore perhaps it would be good
to add some equivalent of "git format-patch" to the TODO file format.


3. Checking rebase

Usually when you are interacting with upstream by sending patches by
email, the last part before sending series of patches is git-rebase
on top of current work.  It would be nice if there were some way to
have "git rebase" (via git-sequencer) to check that all commits
(perhaps with some explicitly stated in TODO file exceptions) passes
pre-commit hook (checking for whitespaces and conflict markers), and
if possibly also either test suite, or relevant parts of test suite.

So perhaps extending TODO format by "check <script>" or
"check-all <script>"?


I'm just not sure if it is worth adding to TOD file format...
-- 
Jakub Narebski
Poland
--

From: Stephan Beyer
Date: Tuesday, July 8, 2008 - 4:49 am

Hi,


I've needed this, too, and I've added this use case in the EXAMPLES
section that will be included in the next patchset.
Basically it is the same as the rebase documentation says:
during a pause (by "edit" or "pause" insn) reset to HEAD^, add only
those parts to the index you want to have in the first patch and commit
and then repeat this.

When I've experienced this need the first time, I thought about how
sequencer could ease that.
A first idea was something like "take only files X, Y, Z from commit ..."
or a --exclude as patch has.
But often splitting a patch is not done file-wise, but hunk-wise or even
line-wise.
So my second idea was to add an option to "pause" that just invokes
the "reset HEAD^". That makes it easy, I think.


That seems to be useful indeed and reminds me of
"git bisect run <script>".
Perhaps it's also better to call it "run" so that it is a generic
way of running scripts from sequencer and if they fail, sequencer
will pause, if they pass, everything goes on.

What about this?


run [--dir=<path>] [--] <cmd> <args>...::
	Run command `<cmd>` with arguments `<args>`.
	Pause (conflict-like) if exit status is non-zero.
+
If `<path>` is set, sequencer will change directory to `<path>`
before running the command and change back after exit.



Regards,
  Stephan


-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F
--

From: Karl
Date: Tuesday, July 8, 2008 - 10:20 pm

(I've hijacked this thread and started talking mostly about StGit
instead of git-sequencer; if you're not interested, you can stop
reading now.)


You can do this without having to manually go to the right patch with
the -p <patchname> flag to stg refresh. In my experimental branch,
this even works together with path limited refresh, or refresh of just
the index.

My own workflow is different: I generally make a large number of
rather small "work-in-progress" commits without much of a commit
message, and every now and then (while I still have everything in
short-term memory) I use "stg coalesce" to make one or more "real"
patches out of them. Because I've committed such small pieces in the

The best way I've found of splitting a patch in StGit is to open the
diff in an Emacs buffer, then pop the patch, and then use Emacs' cool
diff-mode features to apply hunks selectively, split hunks, edit files
in place, etc., and committing at the points where I want patch
boundaries. Occasionally, I'll push and pop the patch to get a new
diff with only the remaining stuff.

I imagine something like this could work without StGit as well, since
it's mostly Emacs doing all the hard work. (And I suppose there are
other tools besides Emacs that can do this?)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Jakub Narebski
Date: Thursday, July 3, 2008 - 6:10 am

First, let's not perpetuate _convenience_ calling convention of "A..B"
of git-diff* family, but use "A B" to set _endpoints_.

Second, with "A B" convention you can fairly easy deal with root
commit, changing the code to the fragment below:

+# Generate message, patch and author script files
+make_patch () {
+	parent_sha1=$(git rev-parse --verify "$1"^) ||
+		echo '--root'
+	git diff-tree -p "$parent_sha1" "$1" >"$PATCH"


BTW. what is best way of checking if given revision is parent-less?
-- 
Jakub Narebski
Poland
ShadeHawk on #git
--

From: Stephan Beyer
Date: Thursday, July 3, 2008 - 2:12 pm

Hi,


I think count the number of parents is quite sane.

In sequencer I have:
	count_parents() {
		git cat-file commit "$1" | sed -n -e '1,/^$/p' | grep -c '^parent'
	}

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F
--

From: Stephan Beyer
Date: Saturday, July 5, 2008 - 9:58 am

git sequencer is planned as a backend for user scripts
that execute a sequence of git instructions and perhaps
need manual intervention, for example git-rebase or git-am.

Mentored-by: Christian Couder <chriscool@tuxfamily.org>
Mentored-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
---
Hi,

this should now contain your suggestions (where I posted interdiffs)
and some further bugfixes of bugs I stumbled across.
(If you prefer an interdiff to v1, I can also send one...)

Regards,
  Stephan

 .gitignore       |    1 +
 Makefile         |    1 +
 command-list.txt |    1 +
 git-sequencer.sh | 1921 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 1924 insertions(+), 0 deletions(-)
 create mode 100755 git-sequencer.sh

diff --git a/.gitignore b/.gitignore
index 4ff2fec..65a075b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -111,6 +111,7 @@ git-revert
 git-rm
 git-send-email
 git-send-pack
+git-sequencer
 git-sh-setup
 git-shell
 git-shortlog
diff --git a/Makefile b/Makefile
index 78e08d3..9db1a43 100644
--- a/Makefile
+++ b/Makefile
@@ -250,6 +250,7 @@ SCRIPT_SH += git-rebase--interactive.sh
 SCRIPT_SH += git-rebase.sh
 SCRIPT_SH += git-repack.sh
 SCRIPT_SH += git-request-pull.sh
+SCRIPT_SH += git-sequencer.sh
 SCRIPT_SH += git-sh-setup.sh
 SCRIPT_SH += git-stash.sh
 SCRIPT_SH += git-submodule.sh
diff --git a/command-list.txt b/command-list.txt
index 3583a33..44bb5b0 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -101,6 +101,7 @@ git-rev-parse                           ancillaryinterrogators
 git-rm                                  mainporcelain common
 git-send-email                          foreignscminterface
 git-send-pack                           synchingrepositories
+git-sequencer                           plumbingmanipulators
 git-shell                               synchelpers
 git-shortlog                            mainporcelain
 git-show                                ...
From: Junio C Hamano
Date: Tuesday, July 1, 2008 - 1:51 am

Heh, these three typically come in different order.  Look at the log
message and docs to see if the design is sound, then look at the code, and
if things overall look Ok, then finally apply and test.

I unfortunately ran out of time tonight and will look at them probably
Thursday evening.  It seems that another GSoC topic is finally nearing
completion and I can shift my attention to other topics now.

By the way Christian is not CC'ed?

--

From: Stephan Beyer
Date: Tuesday, July 1, 2008 - 7:53 am

Hmm, I usually prefer testing first (on such large patches), because I think



Shame on me, I forgot to Cc my mentors ;-)

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F
--

From: Alex Riesen
Date: Friday, July 4, 2008 - 2:00 pm

BTW, how about renaming it in something short: git seq. There is
already a seq(1) in GNU coreutils, which does roughly the same (prints
a sequence of numbers), why not reuse the name?

--

From: Junio C Hamano
Date: Friday, July 4, 2008 - 3:09 pm

Is it advantageous to use shorter but less descriptive name for this
command?  It will be a backend to am/rebase and not something the users
will type from the command line, won't it?
--

From: Stephan Beyer
Date: Friday, July 4, 2008 - 3:23 pm

I also think descriptive names are nice for git even if the user should
type that, since tab completion exists.
When I've started with git, I loved the fact that I could use tab
completion to learn new git commands and that often the name was
descriptive enough to get an imagination of what the tool could do (and

Usually, but for special cases it is also nicely usable by the user
and the --status and --edit subcommands are intended to be used by the
user, too.

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F
--

From: Alex Riesen
Date: Saturday, July 5, 2008 - 1:13 am

There is not a huge lot of possible meanings of "seq" in the given
context. Somehow I find it hard to believe someone will be confused by
a backend command with a short name "seq" (seq-uence-something?)

It'll make the lines shorter, less need to wrap them.

BTW, what does "am" (git am) mean?

--

From: Thomas Adam
Date: Saturday, July 5, 2008 - 3:12 am

I always thought of it as "apply mbox".

-- Thomas Adam
--

From: Johannes Schindelin
Date: Saturday, July 5, 2008 - 3:13 am

Hi,


It might be a bit confusing, since "seq" _produces_ sequences, and 
"sequencer" is kind of an assembly line, getting commits in a sequence and 

By that reasoning, we should have git-a, git-b, ... but that would not 

It means "applymbox", but that name was already taken.  And "am" turned 
out _not_ to replace "applymbox" right away as was expected, so it is a 
bit of unfortunate history.

Ciao,
Dscho

--

Previous thread: [PATCH/RFC] Emulating symlinks on Windows with file copies by Patrick Higgins on Monday, June 30, 2008 - 4:54 pm. (1 message)

Next thread: none