login
Search
It's "worse" than that; rebasing is designed for a _private_ development model. git-rebase is a very handy tool for people like myself (people without a downstream that is) and it basically enables the quilt model of a stack of patches on top of git but public trees that have people pulling from them should generally not rebase or everyone who _is_ pulling finds a different tree each time. Linus is vehement about this also. There have a few threads about it and the most recent was: http://lkml.org/lkml/2008/5/17/190 I'm also not an experienced git user (not other than in my own leafnode developer mode, certainly) but I expect Linus might not terribly mind answering a few questions about the model -- ALSA is a significant subsystem and it switching to GIT might even make for a nice "this is how you do that as a subsystem" treatise... (ie, also added linux-kernel) Rene. --
I don't see big obstacles with this model. You can do changes in your local tree and when 'git pull' fails from the subsystem tree, pull new subsystem tree to a new branch and do rebasing in your local tree, too. Rebasing can keep the subsystem tree more clean I think. It's only about to settle an appropriate workflow. Jaroslav ----- Jaroslav Kysela <perex@perex.cz> Linux Kernel Sound Maintainer ALSA Project, Red Hat, Inc. --
I'm also still frequently trying to figure out an/the efficient way of using GIT but it does seem it's not just a matter of "pure downstream" (which I do believe ALSA has few enough of to not make this be a huge problem). For example linux-next is also going to want to pull in ALSA and say it does, finds a trivial conflict with the trivial tree that it also pulls in and fixes things up. If you rebase that which linux-next pulls from I believe it will have to redo the fix next time it pulls from you since it's getting all those new changesets. I guess this can be avoided by just not rebasing that which linux-next is pulling... and I in fact don't even know if linux-next does any conflict resolution itself, trivial or otherwise. <shrug> I'll see how things work out. Rene. --
At Wed, 21 May 2008 16:40:37 +0200, I thought linux-next does fresh merges at each time, thus it doesn't matter whether a subsystem tree is rebased or not... Takashi --
Let's ask... Fresh merges at each release boundary certainly but if it drops/remerges each subsystem when a bug in its for-next branch is found (a supposedly non rare occurence) all the hopefully hundreds or even thousands of linux-next pullers/testers would seem to have to deal with all those completely new merges everytime as well. I'd hope linux-next during a single release would just pull in the one fix (the subsystem's for-linus branch can still fold it in). Rene. --
Hi Rene, On Wed, 21 May 2008 17:29:56 +0200 Rene Herman <rene.herman@keyaccess.nl> w= Linux-next is rebuilt every day based on Linus' current kernel. I merge all the trees I have been told about and fixup minor conflicts (sometimes reverting commits, sometimes applying patches). So everyday, linux-next is completely new. I do not care if the trees I am merging get rebased. I have only had a couple of occasions when the merge conflicts were so bad that I had to drop a whole tree, but they were fixed up the next day. Linux-next has only one downstream - Andrew's mm tree and he bases on a particular day's linux-next tree each time he rebuilds mm. Testers just need to take the complete tree (which isn't too bad if you are using git since all the linux-next trees share a lot of objects). Does that answer your question? --=20 Cheers, Stephen Rothwell sfr@canb.auug.org.au http://www.canb.auug.org.au/~sfr/
This means you're going to redo all those (same) minor conflict fixups each day again, doesn't it? Or do you pass the fixups back to (one of) But yes, it does, thanks. Rene --
Hi Rene, On Thu, 22 May 2008 22:43:04 +0200 Rene Herman <rene.herman@keyaccess.nl> w= I send email about them and most go away after a day or so i.e. something changes to make the conflicts go away. However, the saving grace here is "git rerere" which remembers these conflicts and their resolutions for me OK, good. --=20 Cheers, Stephen Rothwell sfr@canb.auug.org.au http://www.canb.auug.org.au/~sfr/
At Wed, 21 May 2008 15:04:53 +0200, Theoretically we can work only using merges. However, the resultant tree will look too complex with lots of merge points at the time of the next merge window. This is also a nightmare for bisecting. Thus, most subsystem trees do rebase before the merge window in practice, AFAIK. It'd be appreciated if someone can tell me any good workflow to keep a good-shaped tree without rebasing... Takashi --
Over here: http://lkml.org/lkml/2008/5/17/203 Linus suggests to just not do many merges from upstream -- basically only at releases. I'm only looking at and experiencing things from the limited viewpoint of a leaf node though so I'll shut up for a bit first now as this might not be helpful. Rene. --
At Wed, 21 May 2008 17:40:33 +0200, But, this means that the fixes done outside the subsystem tree cannot be in the subsystem tree itself until the next release. It's a pretty weird situation. Of course, we can do cherry-pick each fix in the upstream, but it doesn't work always, too. Suppose a fix that uses a newly introduced API after the merge window, which doesn't exist in the subsystem tree yet. Or, if you want add new patches during the merge window, etc. The method Linus suggested is suitable for random patches like tirival tree, but apparently not for every case. Takashi --
No it is NOT. Why should you have stuff from outside the subsystem tree? And quite frankly, the only reason to *need* those fixes in the first place is if you merge random test-kernels during the merge window. What you should strive for is to merge at the stable point, not random kernels to keep you "up-to-date", and suddenly you don't need to make more merges just to get (possibly) new fixes. See? If it's the ALSA tree, then what is it doing pulling all the random other stuff that I merge? In other words - merging my random stuff, THAT is the "weird situation". Umm. Bigger subsystems than ALSA are successfully using it and have no problems, and don't think they need to merge backwards: [torvalds@woody linux]$ git rev-list v2.6.25.. drivers/net/ | wc -l 739 [torvalds@woody linux]$ git rev-list v2.6.25.. sound/ | wc -l 291 iow, the only reason you seem to think that you need to merge more is that you merged too much, or from a too-unstable base, to begin with. Aim for basing things on releases, or at least for merging just at stable points (and only when you *need* to, and it will make your life much easier. And the history will automatically be cleaner and less confusing. Linus --
At Wed, 21 May 2008 09:16:05 -0700 (PDT), Well, what I meant is about the fixes to the subsystem (say, ALSA) by people in the outside. Not every ALSA-bugfix patch goes into the upstream from ALSA tree. You, Andrew and others pick individually ALSA-fix patches. They will be missing in the ALSA subsystem tree. And, what if that you need a fix for the fix that isn't in ALSA tree...? IMO, either a rebase or a merge is better than Hmm, that makes me thinking of the development model. We can leave ALSA tree without upstream fixes as is if user always pull both ALSA and upstream trees. This works for users that are Yes, I see the point. But, my question is about the divergence between the development and for-linus branches: how to apply patches that exist only in for-linus tree back. thanks, Takashi --
Well, that's actually fairly rare, but when it happens, either: - if you didn't get the fix (ie you're are just seeing random patches go in that happen to touch alsa), why should you then merge the WHOLE TREE with all my experimental stuff anyway? You can largely ignore it, knowing it's fixed, and when you ask me to pull, we'll have a good end result. - if you got the same fix as a patch, just apply it to your tree (ie just ignore what happens upstream). This happens all the time - people duplicate patches simply because two people apply it. But the real issue is here is that my tree sometimes gets ten THOUSAND commits during the merge window. Do you really want to pull those thousands of commits into your tree just for one or two possible ALSA fixes? In _my_ tree, at least the people involved with asking me to pull end up also having (a) people test it and (b) aware that it's in my tree, so they work on trying to fix it. But if ALSA just merges at random times, neither of those two cases are true. Nobody will know about or test some random state that ALSA merged into its own tree. Ask yourself (and ignore the ALSA parts - think of some totally *different* development area) which you think is better - developing in one area based on a stable base, with the people who do development in that area knowing about that area. - or develop on top of a churning sea of thousands of changes to other sub-areas that you don't know anything about? In other words, the reason I ask people to not do lots of merges is more than just "it looks confusing". It's literally a matter of "it's bad development practice". It causes problems. The confusing history is actually *real* - it's not just a "visual artifact" of looking at the result in gitk. The confusing history is a real phenomenon, and implies First off, I don't see why you even need cherry-picks in the first place. I think your argument is bogus, and you're making it ...
Btw, let me explain this another way.
I do a *lot* of merges as being an "upstream" person. Since 2.6.25, I've
done something like 247 merges (and that's not counting the fast-forward
ones). If you do
git log v2.6.25.. -author=torvalds
you'll see pretty much just merges. It's simply what I do. I have a few
fairly trivial patches every once in a while (although you almost have to
add a "--no-merges" to filter out the merges to see them), but doing
merges is what I do most.
So why would I tell others to not merge, when I've done several hundred
merges in just the last month myself? Isn't that really hypocritical of
me?
The symmetry breaking comes from a few things:
- the merges that "upstream" people do are generally smaller, but even
when they are large, they have a "theme".
Most merges I do are fairly small, but even when they aren't (eg the
network layer merge of all the thousands of commits that were pending
for when the merge window opened), they are _directed_.
IOW, when upstream does a merge, it's hopefully (if the process works
correctly) going to be about a specific sub-area, even if that area may
be pretty big. So the merge has a very specific meaning: "I want to
pull in the changes to subsystem 'xyz'"
In contrast, a merge that goes the other way (subsystem merging
upstream) generally doesn't get any particular directed development
changes, it just gets "everything else".
This also explains why I can do merge summaries, and downstream people
generally can not. Look at my merges, and see how they say things like
Merge git://git.kernel.org/pub/scm/linux/kernel/git/sfrench/cifs-2.6
* git://git.kernel.org/pub/scm/linux/kernel/git/sfrench/cifs-2.6: (21 commits)
[CIFS] Remove debug statement
Fix possible access to undefined memory region.
[CIFS] Enable DFS support for Windows query path info
[CIFS] Enable DFS support for Unix query path info
...one thing that you have missed in your explination in this thread (although you have made the point in other threads) is that subsystem maintainers have the fear that there are other changes that will interfere with their stuff and want to catch it early. per your instructions in prior threads, what they should do is to have a seperate branch on their system that they use as a throw-away branch to pull from your tree, and from their tree to spot problems. As they find problems they can then address them (cherry pick, or whatever) so it's not that the ALSA people should only look at your tree at the merge points, it's that they shouldn't pollute their tree that they are going to publish to you with this checking. David Lang --
Yes. However, that's not just a "my tree" issue. In fact, quite often other trees are more interesting from that angle: for driver subsystems like sound, the changes in Greg's driver core git tree may actually be oftne Yes. Doing throw-away merges is a great way to test not just whether there might be actual merge conflicts, but also to just test that things work together. And even if you want to concentrate your *development* on just ALSA-specific stuff, you may well want to also test all the changes that have gone upstream from other projects (and often do that _together_ with the changes you have developed yourself). And again, for this kind of testing, doing a throw-away merge to see how it all works together is Yes. In general, it's a great idea to have "test trees" that aren't really for development, but for testing. That's obviously what 'linux-next' does, but it's something any tester can do (and it doesn't even have to imply any developer skills, although it would generally require at least some comfort with git). That said, at least as far as I'm concerned, when I pull from some subsystem tree, the thing I really want to know is that the state of that tree is stable on its own. IOW, if the merge itself introduces some subtle bug, that is not only fairly unusual, but it's also something that should not be seen as a bug from the tree I pulled - it's just bad luck. So a submaintainer should care *most* about the fact that his/her tree is stable on its own. Problems that happen when multiple development trees are merged should be the secondary concern. I'd rather have people test their _own_ code really well, than spending lots of time trying to test every possible combination with other peoples trees. Linus --
At Wed, 21 May 2008 11:25:25 -0700 (PDT), Ah, that's what I missed. This suggestion actually makes sense. Takashi --
At Wed, 21 May 2008 10:43:43 -0700 (PDT), Don't get me wrong: I haven't suggested frequent rebases at all. This thread began actually because an update of the present alsa.git tree is required for applying my patches properly. [BACKGROUND: We are trying to make alsa.git tree with multiple committers. And, the current git-rebase doesn't care about sign-offs when a patch was committed by others. But, this is another topic...] However, I have to point that backport or backmerge is a rare case but does happen certainly. For example, assume that we now need to change the codes that touch the device creation. Now on the current your tree, the driver core changed the API. So, we need that change as well. However, picking this particular change might not be enough if it's a part of a long series of patches. BTW, about the stability: we have an independent ALSA tree containing only the subset of the kernel tree (the sound part). On this, we apply patches continuously without rebase or merge. People except for Oh, this is really helpful. Maybe it should be documented somewhere Well, there is another case to think. For example, core API changes or changes of header files. These happen pretty often, at each kernel release, practically :) And, the code I'm working on is for the next kernel release. So, it should follow these changes, too. That is, I need the top-most development tree. This is another "divergence". Or, I could postpone the changes touching these until the next kernel release -- then the tree gets merged anyhow and patches can be applied safely. But, of course, it means the fix or improvement will be Hm, that's what I didn't consider seriously. I thought cherry-picking Yeah, that's exactly what I feel now, too. There is no crystal clear guideline. But, the common sense tells best... Thanks, Takashi --
Cherry-picking can certainly cause merge errors, but not generally very often. Cherry-picking by definition will obviously apply the *same* patch to two different branches, and as a result, when you merge, that merge will generally be totally clean. So a trivial merge that succeeds without you even noticing is actually the common case. But you can certainly get merge failures where you then have to fix things up if there were *other* changes to that same area. At that point, you end up with two different branches that changed the same few lines differently, and it doesn't matter if then _some_ of the changes were identical - the fact that others were not is enough to cause a merge conflict. If cherry-picking is an uncommon situation, the merge problems are not going to show up (and when they do, they'll generally be simple to resolve, especially if you limit cherry-picking to simple fixes). But if you do a *lot* of cherry-picking, and you cherry-pick big changes, then yes, you'll start hitting merge problems. So cherry-picking is fine if you do it (a) fairly seldom and (b) just to small patches, because then the upsides of cherry-picking (easy to get a single fix without merging everything else) are bigger than the downsides (the potential merge problems later). IOW, think of cherry-picking as just another tool. It has upsides and downsides. It's not "wrong" per se, but you can use it the wrong way. You shouldn't use a hammer on a screw, and you shouldn't use cherry-picking for big and complex stuff. Linus --
At Wed, 21 May 2008 12:02:29 -0700 (PDT), Thanks for clarification! Sounds like I should really do this more often to keep the devel tree clean without merge or rebase. Takashi --
With input I used to sync with mainline (pull from Linus) once per release, just before sending a pull request to him. Now I gonna have 3 branches, for-linus and next that I expect to sync only when I need to resolve conflicts or again in merge window and master for -mm which is rebased on top of Linus's tree frequently. We'll see how that works. -- Dmitry --