Re: [StGit PATCH 03/14] Write to a stack log when stack is modified

Previous thread: Re: [PATCH 2/2] git-gc: skip stashes when expiring reflogs by Eric Raible on Wednesday, June 11, 2008 - 9:32 pm. (42 messages)

Next thread: Cleaning up INSTALL before 1.5.6 by Johan Herland on Thursday, June 12, 2008 - 12:20 am. (14 messages)
From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

Now with API documentation added all over the place! Especially in
2/14 and 3/14.

The only two things missing before I'll ask for this to be included
are

  1. Conversion of stg refresh to the new infrastructure, so that it
     can write two separate steps to the stack log (which will allow
     undo et.al. to handle it nicely).

  2. Log writing optimization: Right now, we always write all the
     patches to the log, without trying to reuse the entries for the
     ones that haven't changed since last time.

---

Karl Hasselström (14):
      Make "stg log" show stack log instead of patch log
      Log and undo external modifications
      New command: stg redo
      New command: stg undo
      Move stack reset function to a shared location
      Don't write a log entry if there were no changes
      Add a --hard flag to stg reset
      Log conflicts separately for all commands
      Log conflicts separately
      New command: stg reset
      Add utility function for reordering patches
      Write to a stack log when stack is modified
      Library functions for tree and blob manipulation
      Fix typo


 stgit/commands/branch.py     |   19 +-
 stgit/commands/common.py     |    9 +
 stgit/commands/diff.py       |    2 
 stgit/commands/files.py      |    2 
 stgit/commands/id.py         |    2 
 stgit/commands/log.py        |  169 +++++------------
 stgit/commands/mail.py       |    2 
 stgit/commands/patches.py    |    2 
 stgit/commands/redo.py       |   52 +++++
 stgit/commands/reset.py      |   56 +++++
 stgit/commands/show.py       |    2 
 stgit/commands/status.py     |    3 
 stgit/commands/undo.py       |   49 +++++
 stgit/lib/git.py             |  197 +++++++++++++++++--
 stgit/lib/log.py             |  429 ++++++++++++++++++++++++++++++++++++++++++
 stgit/lib/stack.py           |   16 ++
 stgit/lib/transaction.py     |  121 +++++++++---
 stgit/main.py                |    8 +
 t/t1400-patch-history.sh     |  103 ----------
 t/t3100-reset.sh       ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/lib/git.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/stgit/lib/git.py b/stgit/lib/git.py
index 0e0cccb..6ccdfa7 100644
--- a/stgit/lib/git.py
+++ b/stgit/lib/git.py
@@ -20,7 +20,7 @@ class Immutable(object):
     creating a whole new commit object that's exactly like the old one
     except for the commit message.)
 
-    The L{Immutable} class doesn't acytually enforce immutability --
+    The L{Immutable} class doesn't actually enforce immutability --
     that is up to the individual immutable subclasses. It just serves
     as documentation."""
 

--

From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

Create a log branch (called <branchname>.stgit) for each StGit branch,
and write to it whenever the stack is modified.

Commands using the new infrastructure write to the log when they
commit a transaction. Commands using the old infrastructure get a log
entry write written for them when they exit, unless they explicitly
ask for this not to happen.

The only thing you can do with this log at the moment is look at it.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/branch.py  |   19 ++-
 stgit/commands/common.py  |    8 +
 stgit/commands/diff.py    |    2 
 stgit/commands/files.py   |    2 
 stgit/commands/id.py      |    2 
 stgit/commands/log.py     |    2 
 stgit/commands/mail.py    |    2 
 stgit/commands/patches.py |    2 
 stgit/commands/show.py    |    2 
 stgit/commands/status.py  |    3 -
 stgit/lib/git.py          |    3 -
 stgit/lib/log.py          |  254 +++++++++++++++++++++++++++++++++++++++++++++
 stgit/lib/stack.py        |    9 ++
 stgit/lib/transaction.py  |    3 -
 stgit/main.py             |    2 
 15 files changed, 298 insertions(+), 17 deletions(-)
 create mode 100644 stgit/lib/log.py


diff --git a/stgit/commands/branch.py b/stgit/commands/branch.py
index 50684bb..edbb01c 100644
--- a/stgit/commands/branch.py
+++ b/stgit/commands/branch.py
@@ -25,7 +25,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git, basedir
-
+from stgit.lib import log
 
 help = 'manage patch stacks'
 usage = """%prog [options] branch-name [commit-id]
@@ -40,7 +40,7 @@ When displaying the branches, the names can be prefixed with
 
 If not given any options, switch to the named branch."""
 
-directory = DirectoryGotoToplevel()
+directory = DirectoryGotoToplevel(log = False)
 options = [make_option('-c', '--create',
                        help = 'create a new development branch',
                        action = 'store_true'),
@@ -161,6 +161,7 @@ def func(parser, ...
From: Catalin Marinas
Date: Tuesday, June 17, 2008 - 3:24 am

Hi Karl,

There are some comments below but I think I haven't fully understood this.






I might not fully understand this but can we not store just the commit

Can we not point to the original commit corresponding to the patch? It


Ah, OK. But I think it would be more useful to see the diff between
subsequent revisions of a stack rather than the full patch diff.

Thanks.

-- 
Catalin
--

From: Karl
Date: Tuesday, June 17, 2008 - 5:31 am

I've done my best to answer. Though no doubt you'll have follow-up

I might not have understood precisely what you meant; but I don't
think API backwards compatibilty should be an issue here? I simply fix
all callers. If log should default to true or false is immaterial --
it just means some extra text in one or the other of two equally

"r" in front of a string literal means "raw" (or some such). Escape
sequences aren't recognized inside a raw string -- e.g., r'\n' ==
'\\n'. They are useful when you have to write strings with embedded

Yes. It's stored in a regular git branch. (The design is such that it
should even be possible to pull a stack log from another repository
and _still_ get everything you need.)


You can't store a commit object in a tree. (Well, with submodules you
can, but said commit object isn't protected from gc and won't be
included when pulling.) The idea with this format is that with the two
trees and the info file, you can recreate the patch's commit -- not

Have you tried looking at a patch stack log (in gitk, say)? That is,
"stg log -g" in this patch series. It shows you diffs between
subsequent revisions of the simplified log. I'm sure it's far from
perfect, but I think it's actually quite useful.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Karl
Date: Tuesday, June 17, 2008 - 5:55 am

Oh -- just be sure to use a colorized diff (which you get by default
in gitk, but not in git log, which is what stg log without -g ends up
calling). Without colors, a diff of two diffs is too hard to read, at
least for me.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Catalin Marinas
Date: Tuesday, June 17, 2008 - 7:11 am

Not an issue, I just favour the existing one when the two cases are


But how are the patches recreated when undoing (the
refs/patches/<branch>/* files)? Using the Bottom/Top tree ids that a

What I meant is the SHA1 value of the patch commit instead of
Bottom/Top, Author and Date. The corresponding commit object has all



It is useful, though it might take a bit of time to get used to it. It
might also be a bit difficult if you want to revert some changes to a
single patch but not do a full stack undo which would affect other
patches.

-- 
Catalin
--

From: Karl
Date: Tuesday, June 17, 2008 - 8:32 am

The log actually _contains_ those trees, so there is no problem.

(Relatedly: I've still not written the code that recreates a patch
from the trees and info if its old commit object is no longer present,
but it should be straightforward -- there's even a TODO at the right

I think this question is settled by my answer above: the log actually
contains the trees in question, which it cannot do with the commit
object.

The applied and unapplied files do contain the sha1s of the patches'
commit objects, since most of the time they will still be there, and
we can just re-use them. But gc and pull don't "see" this kind of

You want gitk master.stgit^, which is the simplified log. Can you
guess why I added it? :-)

The full log unfortunately has to look like hell, because it needs
both the stack top and the previous log entry as ancestors (since
ancestry is the only way to point to other commit objects that


Yes, much like diffs take some time to get used to if you haven't seen
them before.


The reset command can already reset just a single patch and not all of
them.

I agree that a "revert"-style command, which undoes just one update
and not everything that happened since then (for the entire stack or
just a single patch), is an intriguing idea. I haven't thought about
it much, but I'm sure it's doable. (In fact, it should be equivalent
to patch stack merging, which I wrote a post about some time ago if
you recall.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Catalin Marinas
Date: Wednesday, June 18, 2008 - 6:03 am

OK, I begin to understand. It is a generic solution for storing
metadata but IMHO it is too overweight when we only need a list of
applied and unapplied files (we can remove hidden) that could be
easily stored in a commit log. It would be useful to run a quick
benchmark with many (hundreds) of patches and compare it with no
logging variant (or the current patch logging, which isn't as
advanced).

Could we not make this (much) simpler? I.e. <branch>.stgit is a commit
object whose tree is <branch>^{tree} and the message contains the
command followed by list of patches in the form "<commit> <patch>"?
This commit can have two parents - the previous <branch>.stgit and
current <branch> head. All the patches are referred via the <branch>
head or the previous <branch> heads if unapplied (assuming that when a
patch is created it is first applied and then popped, maybe this needs
a bit of thinking). This way, a diff between subsequent <branch>.stgit
commits would show the tree changes. The 'stg log' command could be

The diff of diffs is generally useful but for 'refresh' logs, you can
show the diff between the old top tree and the new one (the current
patch log implementation does something like this but it doesn't show
anything for commands like 'push' where a diff of diffs would be more
appropriate).

-- 
Catalin
--

From: Karl
Date: Wednesday, June 18, 2008 - 7:36 am

I don't know about "generic" -- it's made specifically for storing an
StGit stack. You could certainly store just about anything in a git

Do you mean remove hidden patches from StGit altogether, or just not

Will do, as soon as I've done some basic optimization. (A simple
"dirty" flag on each patch will enable us to only write out the log
for the patches that have actually changed, and reuse the rest from

I considered a scheme much like this, but besides the problem with
(the very few) patches that are created unapplied, it fails badly in a
very important corner case: when you start logging in a pre-existing
stack. A similar failure will occur if we ever build some sort of log
pruning (without which the log will grow without limit).

I suppose it would be possible to special-case the very first log
entry: let it have all patches as ancestors. But I don't really see
this format as being simpler than the one I use now: all you've done,
basically, is cram all the data into the commit message instead of
storing it in the tree. (Benchmarking could certainly prove your


The diff-of-diffs view is actually very useful for refresh as well.
The diff of old and new top tree is useful only for refresh (but it
_is_ useful -- I'm not arguing against it).

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Catalin Marinas
Date: Wednesday, June 18, 2008 - 9:16 am

By generic I meant that it is easily extensible to store other blobs
of whatever you need. As you say, a commit message could be extensible

Remove them altogether. Is there anyone using them apart from me? I
could create a "rotten" branch and pick those patches with

It gets too complicated, really. A single commit with the proper
parents could do the job. We could also easily use the commit message

The time to create that tree and blobs worries me a bit, plus the (in
my view) complicated structure.

Making the first log entry special gets difficult with log pruning
(unless you prune the whole log rather than entries older than a
chosen time or number) since you might have to re-create all the
chained log entries as the first log's sha1 will probably change.

The applied patches are chained automatically via HEAD. For unapplied
patches, we could add the correponding commits as parents of the
logging commit (starting with the third parent as the first two are
used for log chaining and applied patches). Do we hit any OS limit
with the number of arguments?

Since we only need the unapplied commits tracking for gc and pull
reasons, we could only create that commit that references them when we
prune the stack log and link it from the top one (but it won't be used
by stgit).

-- 
Catalin
--

From: Karl
Date: Wednesday, June 18, 2008 - 10:32 am

Yes. The problem is that you need quite a lot of parents. Every system
I could come up with that got all the corner cases right was even more

Yes. This is true for any log scheme, though. (But I agree -- longer

You're right to worry. My log format makes things feel slightly slower
when logging a 20-30 deep stack. If I can't make it faster, it's not
viable. But I'm pretty sure I can -- it should be simple to reuse all
the trees and blobs for the patches that weren't touched. (And
operations that touch lots of patches, like rebase, have to write a
lot of git objects anyway as part of their operation, so the relative
slowdown should not be large.)

Anyway, benchmarking is the way to go here. Talk will only get us that

You have to re-create all the commits anyway, since they all are

Not until long after we hit git limits to the number of parents of a
commit. I believe the octopus merge refuses to create merges with more
than about 25 parents, and we probably shouldn't do more than that
either. We'll have to do a tree of octopuses.


Yes, we need to create an "unapplied octopus" if and only if we have
unapplied patches that we can't prove are reachable from the stack top
or the branch head (we have to save both, in case the user has done
something such as git-committing on topp of the stack and caused head
!= top). Which is for the first log entry, and in situations such as
"stg pick --unapplied", but not for "stg pop" and the like.

I do agree that we shouldn't try to use the octopuses to get hold of
the commits, though -- just to keep them reachable. We save the sha1
along with the patch name elsewhere in a more convenient form. (My
proposed format does precisely this.)

So. If I got it right, your proposal is:

  * Tree: just take the HEAD tree.

  * Commit message: list the applied and unapplied patches with their
    commit sha1s.

  * Parents: the previous log entry; branch head; something that
    (recusively) points to all unapplied commits, if ...
From: Catalin Marinas
Date: Thursday, June 19, 2008 - 2:24 am

It would make collaboration using stgit stacks easier. I can start

In the past, I ran some tests because people complained that stgit was
slow compared to quilt:

http://article.gmane.org/gmane.comp.version-control.git/9670

After profiling (the stg-prof file), as expected, most of the time was
spent in external Git calls which I tried to keep to a minimum. With
your approach, you probably add at least 4-5 calls to Git (just a
guess, I haven't counted) and, with a big repository, it will be
visible (I have about 15 branches on my Linux kernel clone going back


For the first log only, we could chain the unapplied patches using
commits with 2 parents. We just need to warn people not to stare at

Yes, but just chain unapplied commits rather than using octopus (you

Yes, but you still have to refer the tree objects corresponding to a


We don't even need to differentiate between applied and unapplied in
the commit message as long as they are listed in order since one of
the parents would represent the boundary between them. Since lib.stack
is pretty well structured, we could later modify PatchOrder to use the

As you pointed below, "branch head" should probably be the "stack
top". We don't need to track the "branch head" if different, just need
to fix up the error and add the patches to the stack. And, anyway, if
one modifies the HEAD using git directly, the log will still point to
the top of the stack.

The third head would only be needed for the first log entry or when we
use pick --unapplied (in the latter, it only points to the unapplied
commit). The third head shouldn't be used at all in the log, just


OK with the version number but the branch head (or stack top) is one

I agree it will look ugly but the simplified log adds an extra
overhead on any stgit action. If we don't use stg log -g, a text only
log command could show the diff. We can add it afterwards though if it
is fast enough.

-- 
Catalin
--

From: Karl
Date: Thursday, June 19, 2008 - 3:07 am

Yes. Collaboration was one of the things I was thinking of when
starting all this. (If not for that, there would be little reason to

The extra git calls take constant time and don't depend on the size of
the repository. But they _are_ a problem. (Which could be solved by
making a new git command specifically for users like StGit that want

How do you imagine we'd do anything except a "full" pruning? There are

We could make chains (or trees) of 16-parent commits -- that'd speed

If there are a lot of unapplied patches, you don't want to throw that

True. But only for patches that have changed. Which will rarely be a

Not if top != head (which I'm convinced we want to allow the design to

Certainly. This has been one of my evil master plans all along. ;-)
Though it would have to be restructured a bit since we can't just
write to a part of the log -- we have to write a complete log entry

If we ever want to be able to undo "stg repair", we have to be able to
represent an inconsistent state where head != top. And it's actually
not difficult -- the patch series I posted recently does this. All it
takes is that we save both the branch head and the stack top, instead

Actually, except for the previous log entry, all the parents are just
there for gc's benefit. So we could just put all of them in the same
bucket -- branch head, stack top, and unapplied patches.

( By "bucket" I mean something like: if there are just a few of them,
  have them as direct parents of the log commit; otherwise, refer to
  them using a tree of octopuses. But in any case, just treat them as
  a set of sha1s that we need to have as ancestors but don't otherwise


I'd actually say the opposite: until we have a good visualizer that
doesn't need the simplified log, we need to have the simplified log.
If I actually have to look at the diffs in the log, I find gitk
indispensible.

Plus, once we try to support merging logs, gitk on a simplified log
will be truly indispensible.

-- 
Karl Hasselström, ...
From: Catalin Marinas
Date: Friday, June 20, 2008 - 2:14 am

I wouldn't bother with this feature. Why would one want to break the
stack again after repairing? If they merge patches and git commits,
they either repair the stack or commit all the patches and continue


And what would the simplified log contain if we decide to go with a
new scheme? In your proposal, it points to the tree of main log and
you get the diff of diffs (which also means that the diffs must be
generated for every modification of a patch). Would this be the same?
Again, I worry a bit about the overhead to generate the patch diff for
every push (with refresh I'm OK). It can be optimised as in the stable
branch where we try git-apply followed by a three-way merge (which,
BTW, I'd like added before 0.15). If git-apply succeeds, there is no
need to re-generate the diff.

-- 
Catalin
--

From: Karl
Date: Monday, June 23, 2008 - 5:36 am

Sometimes, stg repair isn't what you want. Say e.g. that you've done
git merge. stg repair will in this case stop reading your history once
it sees the merge commit, and consider all your patches unapplied. But
what you really want is to undo the git merge. Or that you've done git
rebase -- in that case, stg repair will realize that your patches are
unapplied, which is correct but probably not what you want.

For those of us who know what stg repair does, realizing when not to
run it and what to do instead is easy. But for very little extra
effort in the stack log, we can make the generic "stg undo" able to
fix these mistakes automatically. _And_ give the user assurance that

Yes, I was imagining a simplified log precisely like the one I've
currently implemented (a tree with one blob per patch, which contains
the message, the diff, and some other odds and ends such as the
author). Two optimizations would hopefully make it fast:

  1. If the patch's sha1 hasn't changed, we don't have to regenerate
     the diff.

  2. If the patch's sha1 has changed, but git apply was sufficient
     during the merge stage, we can just reuse that patch. We do have
     to write it to a blob, but we have already generated the diff and
     don't need to do so again. (I've shamelessly stolen your idea
     here.)

In most cases, (1) would make sure that only a small handful of
patches would need to be considered. In the cases where a lot of
patches are touched, such as rebase, (2) would provide a good speedup
(except for the cases where we had to call merge-recursive, and those
are slow anyway).

( By the way: Yes, I agree that we want to try git apply before
  merge-recursive. It's on my TODO list. )

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Catalin Marinas
Date: Saturday, July 12, 2008 - 3:09 am

It can be optimised a bit more to actually apply the diff in the blob
directly rather than the current way of generating the diff (since we

I think it should work. Rebase is indeed my worry but it might be even
faster for most of the patches to apply the blob than computing the
diff. In my experience with the Linux kernel, full merge is rarely
needed.

-- 
Catalin
--

From: Karl
Date: Sunday, July 13, 2008 - 11:32 pm

Ah. Yes, that might be faster. While git is supposedly very fast in

Yes. Now all we need is a good benchmarking system so that we know
we're heading in the right direction. I need to fix up and post that
script I used.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Karl
Date: Tuesday, July 1, 2008 - 1:13 pm

I had an even better idea: no default value. Every caller gets to say
either log = True or log = False, which makes it immediately obvious
to the reader. (That is, every caller still using the old
infrastructure; with the new infrastructure, we log if and only if a
transaction is run.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
--

From: Catalin Marinas
Date: Thursday, July 3, 2008 - 3:05 pm

Fair enough.

-- 
Catalin
--

From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/lib/transaction.py |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)


diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index 4c4da1a..16f5a4b 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -1,6 +1,8 @@
 """The L{StackTransaction} class makes it possible to make complex
 updates to an StGit stack in a safe and convenient way."""
 
+import itertools as it
+
 from stgit import exception, utils
 from stgit.utils import any, all
 from stgit.out import *
@@ -274,3 +276,15 @@ class StackTransaction(object):
             self.__allow_conflicts = lambda trans: True
 
             self.__halt('Merge conflict')
+
+    def reorder_patches(self, applied, unapplied, iw = None):
+        """Push and pop patches to attain the given ordering."""
+        common = len(list(it.takewhile(lambda (a, b): a == b,
+                                       zip(self.applied, applied))))
+        to_pop = set(self.applied[common:])
+        self.pop_patches(lambda pn: pn in to_pop)
+        for pn in applied[common:]:
+            self.push_patch(pn, iw)
+        assert self.applied == applied
+        assert set(self.unapplied) == set(unapplied)
+        self.unapplied = unapplied

--

From: Karl
Date: Wednesday, June 11, 2008 - 10:35 pm

Some commands end up calling log_entry() without verifying that they
did in fact change anything. (One example of this is a conflicting
push, which will log two entries, everything else and the conflicting
push, with the "everything else" part being empty if there was only
one patch to push.) So before appending to the log, make sure that the
entry we're appending isn't a no-op.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/lib/log.py |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/stgit/lib/log.py b/stgit/lib/log.py
index 920c261..3aec6e7 100644
--- a/stgit/lib/log.py
+++ b/stgit/lib/log.py
@@ -164,6 +164,9 @@ def log_entry(stack, msg):
         out.warn(str(e), 'No log entry written.')
         return
     full_log_tree, short_log_tree = log_entry_trees(stack.repository, stack)
+    if len(last_log) == 1 and full_log_tree == last_log[0].full_log.data.tree:
+        # No changes, so there's no point writing a new log entry.
+        return
     stack_log = stack.repository.commit(
         git.CommitData(tree = short_log_tree, message = msg,
                        parents = [ll.stack_log for ll in last_log]))

--

From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

Wrap trees and blobs in Python objects (just like commits were already
wrapped), so that StGit code can read and write them.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/lib/git.py |  188 +++++++++++++++++++++++++++++++++++++++++++++++-------
 1 files changed, 165 insertions(+), 23 deletions(-)


diff --git a/stgit/lib/git.py b/stgit/lib/git.py
index 6ccdfa7..a8881f4 100644
--- a/stgit/lib/git.py
+++ b/stgit/lib/git.py
@@ -182,16 +182,142 @@ class Person(Immutable, Repr):
                 defaults = cls.user())
         return cls.__committer
 
-class Tree(Immutable, Repr):
-    """Represents a git tree object."""
-    def __init__(self, sha1):
+class GitObject(Immutable, Repr):
+    """Base class for all git objects. One git object is represented by at
+    most one C{GitObject}, which makes it possible to compare them
+    using normal Python object comparison; it also ensures we don't
+    waste more memory than necessary."""
+
+class BlobData(Immutable, Repr):
+    """Represents the data contents of a git blob object."""
+    def __init__(self, string):
+        self.__string = str(string)
+    str = property(lambda self: self.__string)
+    def commit(self, repository):
+        """Commit the blob.
+        @return: The committed blob
+        @rtype: L{Blob}"""
+        sha1 = repository.run(['git', 'hash-object', '-w', '--stdin']
+                              ).raw_input(self.str).output_one_line()
+        return repository.get_blob(sha1)
+
+class Blob(GitObject):
+    """Represents a git blob object. All the actual data contents of the
+    blob object is stored in the L{data} member, which is a
+    L{BlobData} object."""
+    typename = 'blob'
+    default_perm = '100644'
+    def __init__(self, repository, sha1):
+        self.__repository = repository
         self.__sha1 = sha1
     sha1 = property(lambda self: self.__sha1)
     def __str__(self):
-        return 'Tree<%s>' % self.sha1
+        return 'Blob<%s>' % ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

Given a commit object from the log, resets the stack (or just the
named patches) to the state given by that log entry.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/reset.py |  116 ++++++++++++++++++++++++++++++++++++
 stgit/main.py           |    2 +
 t/t3100-reset.sh        |  151 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 269 insertions(+), 0 deletions(-)
 create mode 100644 stgit/commands/reset.py
 create mode 100755 t/t3100-reset.sh


diff --git a/stgit/commands/reset.py b/stgit/commands/reset.py
new file mode 100644
index 0000000..b2643b1
--- /dev/null
+++ b/stgit/commands/reset.py
@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+
+__copyright__ = """
+Copyright (C) 2008, Karl Hasselström <kha@treskal.com>
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2 as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+"""
+
+from stgit.commands import common
+from stgit.lib import git, log, transaction
+from stgit.out import out
+
+help = 'reset the patch stack to an earlier state'
+usage = """%prog <state> [<patchnames>]
+
+Reset the patch stack to an earlier state. The state is specified with
+a commit from a stack log; for a branch foo, StGit stores the stack
+log in foo.stgit^. So to undo the last N StGit commands (or rather,
+the last N log entries; there is not an exact one-to-one
+relationship), you would say
+
+  stg reset foo.stgit^~N
+
+or, if you are not sure how many steps to undo, you can ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

This takes care of the old-infrastructure commands as well. They'll
all be converted to the new infrastructure eventually, but until then
this patch is necessary to make all commands behave consistently.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/lib/log.py   |   32 +++++++++++++++++++++++++++++++-
 stgit/lib/stack.py |    7 +++++++
 2 files changed, 38 insertions(+), 1 deletions(-)


diff --git a/stgit/lib/log.py b/stgit/lib/log.py
index 8646e08..920c261 100644
--- a/stgit/lib/log.py
+++ b/stgit/lib/log.py
@@ -174,12 +174,42 @@ def log_entry(stack, msg):
                                   + [ll.full_log for ll in last_log])))
     stack.repository.refs.set(ref, full_log, msg)
 
+class Fakestack(object):
+    """Imitates a real L{Stack<stgit.lib.stack.Stack>}, but with the
+    topmost patch popped."""
+    def __init__(self, stack):
+        appl = list(stack.patchorder.applied)
+        unappl = list(stack.patchorder.unapplied)
+        class patchorder(object):
+            applied = appl[:-1]
+            unapplied = [appl[-1]] + unappl
+            all = appl + unappl
+        self.patchorder = patchorder
+        class patches(object):
+            @staticmethod
+            def get(pn):
+                if pn == appl[-1]:
+                    class patch(object):
+                        commit = stack.patches.get(pn).old_commit
+                    return patch
+                else:
+                    return stack.patches.get(pn)
+        self.patches = patches
+        self.head = stack.head.data.parent
+        self.top = stack.top.data.parent
+        self.base = stack.base
+        self.name = stack.name
+        self.repository = stack.repository
 def compat_log_entry(msg):
     """Write a new log entry. (Convenience function intended for use by
     code not yet converted to the new infrastructure.)"""
     repo = default_repo()
     stack = repo.get_stack(repo.current_branch_name)
-    log_entry(stack, msg)
+    if ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

This patch makes commands that produce a conflict log that final
conflicting push separately from the rest of the command's effects.
This makes it possible for the user to roll back just the final
conflicting push if she desires. (Rolling back the whole operation is
of course still possible, by resetting to the state yet another step
back in the log.)

This change only applies to the new-infrastructure commands.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/lib/transaction.py |   47 +++++++++++++++++++++++++++++++---------------
 1 files changed, 32 insertions(+), 15 deletions(-)


diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index 16f5a4b..6347b14 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -79,6 +79,7 @@ class StackTransaction(object):
         self.__patches = _TransPatchMap(stack)
         self.__applied = list(self.__stack.patchorder.applied)
         self.__unapplied = list(self.__stack.patchorder.unapplied)
+        self.__conflicting_push = None
         self.__error = None
         self.__current_tree = self.__stack.head.data.tree
         self.__base = self.__stack.base
@@ -160,19 +161,26 @@ class StackTransaction(object):
             out.error(self.__error)
 
         # Write patches.
-        for pn, commit in self.__patches.iteritems():
-            if self.__stack.patches.exists(pn):
-                p = self.__stack.patches.get(pn)
-                if commit == None:
-                    p.delete()
+        def write(msg):
+            for pn, commit in self.__patches.iteritems():
+                if self.__stack.patches.exists(pn):
+                    p = self.__stack.patches.get(pn)
+                    if commit == None:
+                        p.delete()
+                    else:
+                        p.set_commit(commit, msg)
                 else:
-                    p.set_commit(commit, self.__msg)
-            else:
-                self.__stack.patches.new(pn, ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:34 pm

With this flag, reset will overwrite any local changes. Useful e.g.
when undoing a push that has polluted the index+worktree with a heap
of conflicts.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/reset.py  |   13 +++++++----
 stgit/lib/git.py         |    4 +++
 stgit/lib/transaction.py |   17 ++++++++++++--
 t/t3101-reset-hard.sh    |   56 ++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 82 insertions(+), 8 deletions(-)
 create mode 100755 t/t3101-reset-hard.sh


diff --git a/stgit/commands/reset.py b/stgit/commands/reset.py
index b2643b1..a7b5d35 100644
--- a/stgit/commands/reset.py
+++ b/stgit/commands/reset.py
@@ -17,12 +17,13 @@ along with this program; if not, write to the Free Software
 Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 """
 
+from optparse import make_option
 from stgit.commands import common
 from stgit.lib import git, log, transaction
 from stgit.out import out
 
 help = 'reset the patch stack to an earlier state'
-usage = """%prog <state> [<patchnames>]
+usage = """%prog [options] <state> [<patchnames>]
 
 Reset the patch stack to an earlier state. The state is specified with
 a commit from a stack log; for a branch foo, StGit stores the stack
@@ -43,9 +44,10 @@ If one or more patch names are given, reset only those patches, and
 leave the rest alone."""
 
 directory = common.DirectoryHasRepositoryLib()
-options = []
+options = [make_option('--hard', action = 'store_true',
+                       help = 'discard changes in your index/worktree')]
 
-def reset_stack(stack, iw, state, only_patches):
+def reset_stack(stack, iw, state, only_patches, hard):
     only_patches = set(only_patches)
     def mask(s):
         if only_patches:
@@ -55,7 +57,7 @@ def reset_stack(stack, iw, state, only_patches):
     patches_to_reset = mask(set(state.applied + state.unapplied))
     existing_patches = set(stack.patchorder.all)
     to_delete = mask(existing_patches - ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:35 pm

Command for undoing an undo.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/redo.py |   21 +++++++++------
 stgit/lib/log.py       |   21 ++++++++++++---
 stgit/main.py          |    2 +
 t/t3104-redo.sh        |   66 +++++++++++++++++++++++++++++++++++++-----------
 4 files changed, 81 insertions(+), 29 deletions(-)
 copy stgit/commands/{undo.py => redo.py} (71%)
 copy t/{t3102-undo.sh => t3104-redo.sh} (53%)


diff --git a/stgit/commands/undo.py b/stgit/commands/redo.py
similarity index 71%
copy from stgit/commands/undo.py
copy to stgit/commands/redo.py
index b1d7de9..47221a5 100644
--- a/stgit/commands/undo.py
+++ b/stgit/commands/redo.py
@@ -19,28 +19,31 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 from optparse import make_option
 from stgit.commands import common
-from stgit.lib import git, log, transaction
-from stgit.out import out
+from stgit.lib import log, transaction
 
-help = 'undo the last operation'
+help = 'undo the last undo operation'
 usage = """%prog [options]
 
-Reset the patch stack to the previous state. Consecutive invocations
-of "stg undo" will take you ever further into the past."""
+If the last command was an undo, reset the patch stack to the state it
+had before the undo. Consecutive invocations of "stg redo" will undo
+the effects of consecutive invocations of "stg undo".
+
+It is an error to run "stg redo" if the last command was not an
+undo."""
 
 directory = common.DirectoryHasRepositoryLib()
 options = [make_option('-n', '--number', type = 'int', metavar = 'N',
                        default = 1,
-                       help = 'undo the last N commands'),
+                       help = 'undo the last N undos'),
            make_option('--hard', action = 'store_true',
                        help = 'discard changes in your index/worktree')]
 
 def func(parser, options, args):
     stack = directory.repository.current_stack
     if options.number < 1:
-        ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:35 pm

Basically, this is just a user-friendly way to access a subset of the
functionality of "stg reset".

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/undo.py  |   40 +++++++---------------
 stgit/lib/log.py        |   38 +++++++++++++++++++++
 stgit/main.py           |    2 +
 t/t3102-undo.sh         |   86 +++++++++++++++++++++++++++++++++++++++++++++++
 t/t3103-undo-hard.sh    |   10 +++--
 5 files changed, 144 insertions(+), 32 deletions(-)
 copy stgit/commands/{reset.py => undo.py} (56%)
 create mode 100755 t/t3102-undo.sh
 copy t/{t3101-reset-hard.sh => t3103-undo-hard.sh} (82%)


diff --git a/stgit/commands/reset.py b/stgit/commands/undo.py
similarity index 56%
copy from stgit/commands/reset.py
copy to stgit/commands/undo.py
index 5ad9914..b1d7de9 100644
--- a/stgit/commands/reset.py
+++ b/stgit/commands/undo.py
@@ -22,42 +22,28 @@ from stgit.commands import common
 from stgit.lib import git, log, transaction
 from stgit.out import out
 
-help = 'reset the patch stack to an earlier state'
-usage = """%prog [options] <state> [<patchnames>]
+help = 'undo the last operation'
+usage = """%prog [options]
 
-Reset the patch stack to an earlier state. The state is specified with
-a commit from a stack log; for a branch foo, StGit stores the stack
-log in foo.stgit^. So to undo the last N StGit commands (or rather,
-the last N log entries; there is not an exact one-to-one
-relationship), you would say
-
-  stg reset foo.stgit^~N
-
-or, if you are not sure how many steps to undo, you can view the log
-with "git log" or gitk
-
-  gitk foo.stgit^
-
-and then reset to any sha1 you see in the log.
-
-If one or more patch names are given, reset only those patches, and
-leave the rest alone."""
+Reset the patch stack to the previous state. Consecutive invocations
+of "stg undo" will take you ever further into the past."""
 
 directory = common.DirectoryHasRepositoryLib()
-options = [make_option('--hard', action = 'store_true',
+options = ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:35 pm

Move reset_stack() from commands/reset.py to lib/log.py, so that more
commands besides reset can use it. (No such commands exist currently,
but undo and redo will use it.)

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/reset.py  |   70 +++++-----------------------------------------
 stgit/lib/log.py         |   62 +++++++++++++++++++++++++++++++++++++++++
 stgit/lib/transaction.py |    3 +-
 3 files changed, 71 insertions(+), 64 deletions(-)


diff --git a/stgit/commands/reset.py b/stgit/commands/reset.py
index a7b5d35..5ad9914 100644
--- a/stgit/commands/reset.py
+++ b/stgit/commands/reset.py
@@ -47,67 +47,6 @@ directory = common.DirectoryHasRepositoryLib()
 options = [make_option('--hard', action = 'store_true',
                        help = 'discard changes in your index/worktree')]
 
-def reset_stack(stack, iw, state, only_patches, hard):
-    only_patches = set(only_patches)
-    def mask(s):
-        if only_patches:
-            return s & only_patches
-        else:
-            return s
-    patches_to_reset = mask(set(state.applied + state.unapplied))
-    existing_patches = set(stack.patchorder.all)
-    to_delete = mask(existing_patches - patches_to_reset)
-    trans = transaction.StackTransaction(stack, 'reset', discard_changes = hard)
-
-    # If we have to change the stack base, we need to pop all patches
-    # first.
-    if not only_patches and trans.base != state.base:
-        trans.pop_patches(lambda pn: True)
-        out.info('Setting stack base to %s' % state.base.sha1)
-        trans.base = state.base
-
-    # In one go, do all the popping we have to in order to pop the
-    # patches we're going to delete or modify.
-    def mod(pn):
-        if only_patches and not pn in only_patches:
-            return False
-        if pn in to_delete:
-            return True
-        if stack.patches.get(pn).commit != state.patches.get(pn, None):
-            return True
-        return False
-    ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:35 pm

At the beginning of every StGit command, quickly check if the branch
head recorded in the log is the same as the actual branch head; if
it's not, conclude that some non-StGit tool has modified the stack,
and record a log entry that says so. (Additionally, if the log doesn't
exist yet, create it.)

This introduces the possibility that a log entry specifies a head and
a top that aren't equal. So teach undo, redo, and reset to deal with
that case.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/common.py     |    1 
 stgit/commands/redo.py       |    4 +-
 stgit/commands/reset.py      |    8 +++-
 stgit/commands/undo.py       |    4 +-
 stgit/lib/log.py             |   88 ++++++++++++++++++++++++++++--------------
 stgit/lib/transaction.py     |   39 ++++++++++++-------
 t/t3105-undo-external-mod.sh |   68 ++++++++++++++++++++++++++++++++
 7 files changed, 162 insertions(+), 50 deletions(-)
 create mode 100755 t/t3105-undo-external-mod.sh


diff --git a/stgit/commands/common.py b/stgit/commands/common.py
index fd02398..2689a42 100644
--- a/stgit/commands/common.py
+++ b/stgit/commands/common.py
@@ -525,6 +525,7 @@ class DirectoryAnywhere(_Directory):
 class DirectoryHasRepository(_Directory):
     def setup(self):
         self.git_dir # might throw an exception
+        log.compat_log_external_mods()
 
 class DirectoryInWorktree(DirectoryHasRepository):
     def setup(self):
diff --git a/stgit/commands/redo.py b/stgit/commands/redo.py
index 47221a5..a1075ec 100644
--- a/stgit/commands/redo.py
+++ b/stgit/commands/redo.py
@@ -46,7 +46,7 @@ def func(parser, options, args):
     trans = transaction.StackTransaction(stack, 'redo %d' % options.number,
                                          discard_changes = options.hard)
     try:
-        log.reset_stack(trans, stack.repository.default_iw, state, [])
+        log.reset_stack(trans, stack.repository.default_iw, state)
     except transaction.TransactionHalted:
         pass
-    return ...
From: Karl
Date: Wednesday, June 11, 2008 - 10:35 pm

Make "stg log" show the new stack log instead of the old patch logs,
which is now obsolete. Delete t1400-patch-history, which is specific
to the old "stg log".

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/log.py    |  169 ++++++++++++++--------------------------------
 stgit/commands/reset.py  |   15 +---
 stgit/lib/log.py         |    3 +
 t/t1400-patch-history.sh |  103 ----------------------------
 4 files changed, 58 insertions(+), 232 deletions(-)
 delete mode 100755 t/t1400-patch-history.sh


diff --git a/stgit/commands/log.py b/stgit/commands/log.py
index 13e0baa..cf15c7d 100644
--- a/stgit/commands/log.py
+++ b/stgit/commands/log.py
@@ -1,5 +1,8 @@
+# -*- coding: utf-8 -*-
+
 __copyright__ = """
 Copyright (C) 2006, Catalin Marinas <catalin.marinas@gmail.com>
+Copyright (C) 2008, Karl Hasselström <kha@treskal.com>
 
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License version 2 as
@@ -15,133 +18,67 @@ along with this program; if not, write to the Free Software
 Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 """
 
-import sys, os, time
-from optparse import OptionParser, make_option
-from pydoc import pager
-from stgit.commands.common import *
-from stgit import stack, git
-from stgit.out import *
-from stgit.run import Run
+import os.path
+from optparse import make_option
+from stgit import run
+from stgit.commands import common
+from stgit.lib import log
+from stgit.out import out
 
 help = 'display the patch changelog'
-usage = """%prog [options] [patch]
-
-List all the current and past commit ids of the given patch. The
---graphical option invokes gitk instead of printing. The changelog
-commit messages have the form '<action> <new-patch-id>'. The <action>
-can be one of the following:
+usage = """%prog [options] [<patchnames>]
 
-  new     - new patch created
-  refresh - local changes were added to the patch
-  push  ...
Previous thread: Re: [PATCH 2/2] git-gc: skip stashes when expiring reflogs by Eric Raible on Wednesday, June 11, 2008 - 9:32 pm. (42 messages)

Next thread: Cleaning up INSTALL before 1.5.6 by Johan Herland on Thursday, June 12, 2008 - 12:20 am. (14 messages)