Hello everyone,

Thanks to help from David Woodhouse and Linus, I have btrfs source repos
migrated over to git.  My goal is to make the sources easier for people
to review, and to start some discussions around the best time to merge
Btrfs.

The current code has a number of problems (and this is not a complete
list):

* The disk format is not finalized
* ENOSPC can result in BUG()
* blocksize != pagesize does not work
* Error handling is missing in a number of places

But, the code is very actively developed, and I believe the best way to
develop Btrfs from here is to get it into the mainline kernel (with a
large warning label about the disk format) and attract more extensive
review of both the disk format and underlying code.

The Btrfs developers are committed to making the FS work and to working
well within the kernel community.  I think everyone will be happier with
the final result if I am able to attract eyeballs as early as possible.

The plan is to have the disk format finalized by the end of the year.  I
do expect to have a provisional disk format over the next 2 weeks that
should have all the moving pieces btrfs needs to implement the remaining
1.0 features, and I hope to restrict any additional format changes after
that to include backward compatibility.  

The sources:

Kernel module (against rc7):
http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-unstable.git;a=summary


Btrfs progs:

http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs-unstable.git;a=summary

General Btrfs information:

http://btrfs.wiki.kernel.org/

Most of the remaining features before 1.0:

http://btrfs.wiki.kernel.org/index.php/Development_timeline

-chris


--

[ message continues ]

For the record...  I have been encouraging Chris to get btrfs into
mainline soon.  Get it into linux-next asap and merge it into 2.6.29.

And do this even though the on-disk format is still changing - we emit a
loud printk at mount time and if someone comes to depend upon some
intermediate format, well, that's their tough luck.

My thinking here is that btrfs probably has a future, and that an early
merge will accelerate its development and will broaden its developer base. 
If it ends up failing for some reason, well, we can just delete it
again.


For various reasons this approach often isn't appropriate as a general
policy thing, but I do think that Linux has needed a new local
filesystem for some time, and btrfs might be The One, and hence is
worth a bit of special-case treatment.

--

[ message continues ]

Let's try to learn from the past:

6 days from today ext4 (another new local filesystem for Linux) 
celebrates the second birthday of it's inclusion into Linus' tree
as a similar special-case.

You claim "an early merge will accelerate its development and will 
broaden its developer base" for Btrfs.

Read the timeline Ted outlined back in June 2006 for ext4 [1].
When comparing with what happened in reality it kinda disproves
your "acceleration" point.

cu
Adrian

[1] http://lkml.org/lkml/2006/6/28/454

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

--

[ message continues ]

OTOH, maybe it's just me, but I think there is more excitement around
btrfs.  Myself I'm dying for snapshot support, and can't wait to try
btrfs on a separate data/scratch partition (where i don't mind losing
data).  btrfs and nilfs - yay.  Ext4?  <yawn>  That can make all the
difference.

-serge
--

[ message continues ]

"accelerate its development and will broaden its developer base" is not 
about users/testers but about people doing code development.

For people wanting to try WIP code you don't need it in mainline.

Stable kernels will anyway usually contain months old code of the
WIP filesystem that is not usable for testing, so for any meaningful
testing you will still have to follow the btrfs tree and not mainline.

This is not meant as a statement on the quality of ext4 or btrfs, or any 
comparison of the development times of ext4 and btrfs, but for ext4 the 
advantages Andrew thinks would happen with an early btrfs merge do not 
seem to have happened.

I just realize that I forgot to add Ted and the ext4 mailing list into 
the Cc of my first email. Adding them to the Cc, so if I'm talking 

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

--

[ message continues ]

The btrfs timelines have always been aggressive, and as btrfs gets
closer to feature complete, the testing matrix grows dramatically.  I
can't promise my crazy timelines won't slip, but I've been hacking away
in the basement for almost 18 months now and it's time for me to get off
the pot and make it stable.

Ext4 has always had to deal with the ghost of ext3.  Both from a
compatibility point of view and everyone's expectations of stability.  I
believe that most of us underestimated how difficult it would be to move
ext4 forward.

Btrfs is different for lots of reasons, and being in mainline will
definitely increase the pressure on the btrfs developers to finish, and

People want btrfs for different reasons.  I want btrfs in the kernel
because when you're in the kernel more people look at it, and when
people look at it they send me email with the mistakes they found.

For example, see the streaming write patches I sent to fsdevel last
week.  I wouldn't test against ext4 as often if I had to hunt down
external repos just to get something consistent with the current
development kernels.  ext4 in mainline makes it much easier for me to

For ext4 at least, the mainline code is very usable.  I hope to have
btrfs in shape for that by the 2.6.29 merge cycle.

-chris


--

[ message continues ]

Your last sentence does not make sense:

According to your timeline btrfs 1.0 will be released in Q408 [1] - and

One risk you should be aware of is that when btrfs is in 2.6.29 part of 
the Linux press might pick it up and stress test and benchmark this new 
filesystem.

JFS still suffers from from not being that good when it was

cu
Adrian

[1] http://btrfs.wiki.kernel.org/index.php/Development_timeline

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

--

[ message continues ]

Planning for mainline inclusion is always a guessing game.  Cutting 1.0
is different from being in mainline, and the dates don't have to be the

I think the gains from early testing far outweigh the risks of bad early
press.

-chris


--

[ message continues ]

The 2.6.29 merge window will start in January - everything else is much
more unlikely than an incompetent chick from Alaska getting only one 

Sure, but Andrew's "special-case treatment" suggestion does not apply if 

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

--

[ message continues ]

These comments are inappropriate for this list.
--

[ message continues ]

and you dropped several mailing list cc's.  Please don't do that.

-- 
~Randy
--

[ message continues ]

Why?

--

[ message continues ]

I was talking about the Alaska chick thing.  The comment is wrong, but I 
didn't object to his politics, just the fact it was on the kernel developer's 
list.  The comments about the merge window are of course on topic.
--

[ message continues ]

But it saves time for the user, who does not have to run around chasing
links, carefully checking for a kernel match, downloading, patching,
building and installing a single purpose kernel, and bringing it up on
a machine that would probably have only required one click on the new
filesystem option otherwise.  The considerable time thus saved can be

True, but the trick here is getting started.  It is much easier to
justify the effort of going out and getting the latest patch if one

Are you sure about that?  I see 33 messages on linux-ext4 yesterday,
from a broad range of contributors.  Versus eight from a much narrower
range of contributors, Oct 4 a year ago.

There is little question that an early merge helps both developers and
users employ their time more efficiently, once a project is past the
point where we wonder about its value and/or viability.  In my opinion,
Btrfs clearly has both.  Particularly because we need a way to stem the
loss of mindshare to ZFS in the storage space, which is significant at
the moment.  And Btrfs is closest to the finish line in that regard.
It needs all the help it can get.

Regards,

Daniel
--

[ message continues ]

Bug reports against a 3-6 months old snapshot of a filesystem being 
under heavy development.

Ted said back in August in the announcement of an ext4 patchset:

"As before I've also released updated the patch set vs. the 2.6.26 stock
 kernel, for those people who don't want to play with development 
 kernels but who still want to test out ext4." [1]

When running stable kernels you still have to patch, build and install

There are lies, damn lies, and statistics.  ;)

Single day statistics about mailing list postings are not very good
indicators for anything.

And since linux-ext4 is for all of ext2/ext3/ext4 the data you gave 
could equally be used to prove that ext3 recently became much more

Regarding users see my comment above.

Regarding developers it would be interesting to hear some experiences 
from ext4 developers about their experiences (or get a pointer to them 

2 years ago ext4 was in a similar situation of being regarded as an 

cu
Adrian

BTW: My comments are not in any way meant against btrfs or ext4.
     I just question the advantages of merging them early.


[1] http://lwn.net/Articles/294784/

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

--

[ message continues ]

I've refrained from commenting on this thread mainly because I've been
hugely busy between (a) Linux Foundation 2009 budget planning, (b)
preparing for the Linux Foundation end user summit, (c) making sure
ext4 tree was ready for the opening of the 2.6.28 merge window, and a
million other things --- and because I think this thread is largely
pointless.  At the end of the day, it's mostly Adrian arguing against
early merging, and if Andrew is favor of merging btrfs this point, and
given Linus preferences for early merging of things like device
drivers, it's going to happen regardless of Adrian's opinions.

From the point of the filesystem, it's all upside to be merged into
the kernel mainline.  We've *always* said that we strongly discourage
out-of-tree kernel modules, whether it's device drivers or externally
maintained filesystems whether it's binary VxFS or ClearCase
filesystem (which is GPL'ed and yet maintained out of tree for a
variety of reasons).  One of the things that we do in order to
strongly discourage out of tree drivers/filesystems is that we
constantly make changes to the API, without regard to making life easy
to the out-of-tree kernel module.  Most of the time the changes are
justified --- although sometimes people have suspected that some
changes made had benefits that were so marginal that it seemed that
the main justification was to screw over externally maintained
drivers/filesystems.  Whether or not that's true, the official party
line is that we show no mercy towards externally maintained device
drivers (even ones that are GPL'ed); the Right Answer is that they
should be part of the mainline kernel.

If that is true, there are very few justifications for keeping a
proposed kernel module out of the tree.  The main consideration is
whether the code will, in the long term, be maintainable.  There is
some minimum level of quality that is needed, although there is some
disagreement about what that level is; but probably what is more
important is the reputation ...
[ message continues ]

Definitely. While, due to the changing disk format, long-term storage
is not really an option yet, the filesystem can be really useful in
network-booted hybrid clients that have a local disk in an unionfs
compound mount as a COW rw storage device (because RAM *is* limited
after all). Keep up the work.


Jan
--

[ message continues ]