Reading through the lengthy debate on the lkml titled "silent semantic changes with reiser4" [story] is a time investment. Comprised of well over 500 emails and growing, I include here a tiny snippet containing a discussion primarily between Hans Reiser, Andrew Morton [interview], and Linus Torvalds. Questions raised include whether or not the filesystem should be ultimately merged into the mainline kernel, and if so how to go about this. Much of the debate is regarding extensions that are currently only available through reiser4, and perhaps not fully compatible with existing utilities. The thread within begins with some coments by Andrew, who suggests that if the provided feature set is the desired direction for the Linux kernel, his preference would be to "accept the reiser4-only extensions with a view to turning them into kernel-wide extensions at some time in the future, so all filesystems will offer the extensions (as much as possible)".

As quoted earlier [story], Hans stressed that it was important that the reiser4 functionality be merged so that Linux is capable of competing with WinFS and Spotlight. The argument was continued by others, and to these followup comments Linus retorted:

"Hell will freeze over before Microsoft does a filesystem right. Besides, WinFS is likely almost in user mode anyway, ie mostly a library, rather like the gnome people are already doing with gnome storage. So there's really no point in trying to push your agenda by trying to scare people with MS activities. Linux kernel developers do what's right because it is _right_, not because somebody else does it."

From: Andrew Morton [email blocked]
To: Hans Reiser [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Wed, 25 Aug 2004 15:28:05 -0700

Hans Reiser [email blocked] wrote:
>
> I had not intended to respond to this because I have nothing positive to 
> say, but Andrew said I needed to respond and suggested I should copy 
> Linus.

Yes, but I didn't say "flame Christoph and ignore the issues" ;)

There are lots of little things to do with implementation, coding style,
module exports, deadlocks, what code goes where, etc.  These are all normal
daily kernel business and we should set them aside for now and concentrate
on the bigger issues.

And as I see it, there are two big issues:

a) reiser4 extends the Linux API in ways which POSIX/Unix/etc do not
   anticipate and 

b) it does this within the context of just a single filesystem.

I see three possible responses:

a) accept the reiser4-only extensions as-is (possibly with post-review
   modifications, of course) or

b) accept the reiser4-only extensions with a view to turning them into
   kernel-wide extensions at some time in the future, so all filesystems
   will offer the extensions (as much as poss) or

c) reject the extensions.


My own order of preference is b) c) a).  The fact that one filesystem will
offer features which other filesystems do not and cannot offer makes me
queasy for some reason.

b) means that at some time in the future we need to hoist the reiser4
extensions (at a conceptual level) (and probably with restrictions) up into
the VFS.  This will involve much thought, argument and work.


To get us started on this route it would really help me (and, probably,
others) if you could describe what these API extensions are in a very
simple way, without referring to incomprehehsible web pages, and without
using terms which non-reiser4 people don't understand.

It would be best if each extension was addressed in a separate email
thread.

We also need to discuss what a reiser4 "module" is, what its capabilities
are, and what licensing implications they have.

Then, we can look at each one and say "yup, that makes sense - we want
Linux to do that" and we can also think about how we would implement it at
the VFS level.

If we follow the above route I believe we can make progress in a technical
direction and not get deadlocked on personal/political stuff.


Now, an alternative to all the above is to just merge reiser4 as-is, after
addressing all the lower-level coding issues.  And see what happens.  That
may be a thing which Linus wishes to do.  I'm easy.



From: Linus Torvalds [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Wed, 25 Aug 2004 15:59:43 -0700 (PDT)



On Thu, 26 Aug 2004, Spam wrote:
> 
> > My own order of preference is b) c) a).  The fact that one filesystem will
> > offer features which other filesystems do not and cannot offer makes me
> > queasy for some reason.
> 
>   This last sentence makes me wonder. Where is Linux heading? The idea
>   that   a  FS  cannot  contain  features that no other FS has is very
>   scary.

That's not what Andrew said or meant.

Note the "cannot offer". As in "there is no way to offer them even if the 
filesystem could support it otherwise". 

We have tons of filesystems that do things other filesystems cannot do.  
Most filesystems support writing to a file - despite the fact that some
filesystems (iso9600 being an obvious one) cannot. The infrastructure is
there in the VFS layer, and it becomes a _choice_ for the filesystem
whether it offers certain capabilities.

So look at what Andrew said, again: his top choice would be (b). Let's see 
what that was again, shall we?

> b) accept the reiser4-only extensions with a view to turning them into
>    kernel-wide extensions at some time in the future, so all filesystems
>    will offer the extensions (as much as poss) or

In other words, if reiserfs does something special, we should make 
standard interfaces for doing that special thing, so that everybody can 
do it without stepping on other peoples toes.

That doesn't mean that we'd _force_ everybody to do it. The same way we 
don't force iso9660 to write to a CD-ROM.

		Linus



From: Andrew Morton [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Wed, 25 Aug 2004 16:32:25 -0700

Spam [email blocked] wrote:
>
> > In other words, if reiserfs does something special, we should make 
> > standard interfaces for doing that special thing, so that everybody can
> > do it without stepping on other peoples toes.
> 
>   Agreed  that would be the best. But how much time and effort will it
>   be

Zero.

We can add these new features tomorrow, as reiser4-only features, with a
plan in hand to generalise them later.

-->>__if__<<-- we think these are features which Linux should offer.


From: Hans Reiser [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Thu, 26 Aug 2004 01:31:34 -0700

Andrew Morton wrote:

>Hans Reiser [email blocked] wrote:
>  
>
>>I had not intended to respond to this because I have nothing positive to 
>>say, but Andrew said I needed to respond and suggested I should copy 
>>Linus.
>>    
>>
>
>Yes, but I didn't say "flame Christoph and ignore the issues" ;)
>  
>
Oh....;-)

>There are lots of little things to do with implementation, coding style,
>module exports, deadlocks, what code goes where, etc.  These are all normal
>daily kernel business and we should set them aside for now and concentrate
>on the bigger issues.
>  
>
Yes, you are right, but I am not sure Viro will go along with that..... ;-)

>And as I see it, there are two big issues:
>
>a) reiser4 extends the Linux API in ways which POSIX/Unix/etc do not
>   anticipate and 
>
>b) it does this within the context of just a single filesystem.
>
>I see three possible responses:
>
>a) accept the reiser4-only extensions as-is (possibly with post-review
>   modifications, of course) or
>
>b) accept the reiser4-only extensions with a view to turning them into
>   kernel-wide extensions at some time in the future, so all filesystems
>   will offer the extensions (as much as poss) or
>
>c) reject the extensions.
>
>
>My own order of preference is b) c) a). 
>
I don't object to b), though I think b) should wait for 2.7 and reiser4 
should not.

> The fact that one filesystem will
>offer features which other filesystems do not and cannot offer makes me
>queasy for some reason.
>  
>
Andrew, we need to compete with WinFS and Dominic Giampaolo's filesystem 
for Apple, and that means we need to put search engine and database 
functionality into the filesystem.  It takes 11 years of serious 
research to build a clean storage layer able to handle doing that.  
Reiser4 has done that, finally.  None of the other Linux filesystems 
have.  The next major release of ReiserFS is going to be bursting with 
semantic enhancements, because the prerequisites for them are in place 
now.  None of the other Linux filesystems have those prerequisites.  
They won't be able to keep up with the semantic enhancements.  This 
metafiles and file-directories stuff is actually fairly trivial stuff.

Look guys, in 1993 I anticipated the battle would be here, and I build 
the foundation for a defensive tower right at the spot MS and Apple are 
now maneuvering towards.  Help me get the next level on the tower before 
they get here.  It is one hell of a foundation, they won't be able to 
shake it, their trees are not as powerful.  Don't move reiser4 into vfs, 
use reiser4 as the vfs.  Don't write filesystems, write file plugins and 
disk format plugins and all the other kinds of plugins, and you won't be 
missing any expressive power that you really want....

Give Saveliev and I some credit.  10 years of hard work at an ivory 
tower nobody thought mattered.  Now the battle leaves the browser and 
swings our way.  Don't duplicate that infrastructure, use it. 

There is so much we could use help with if talented people like you 
chose to contribute.

>b) means that at some time in the future we need to hoist the reiser4
>extensions (at a conceptual level) (and probably with restrictions) up into
>the VFS.  This will involve much thought, argument and work.
>
>
>To get us started on this route it would really help me (and, probably,
>others) if you could describe what these API extensions are in a very
>simple way, without referring to incomprehehsible web pages,
>
what is not comprehensible....?

> and without
>using terms which non-reiser4 people don't understand.
>  
>
Well, I agree that there is value in defining things in more detail than 
we have.

>It would be best if each extension was addressed in a separate email
>thread.
>
>We also need to discuss what a reiser4 "module" is, what its capabilities
>are, and what licensing implications they have.
>
>Then, we can look at each one and say "yup, that makes sense - we want
>Linux to do that" and we can also think about how we would implement it at
>the VFS level.
>
>If we follow the above route I believe we can make progress in a technical
>direction and not get deadlocked on personal/political stuff.
>
>
>Now, an alternative to all the above is to just merge reiser4 as-is, after
>addressing all the lower-level coding issues.  And see what happens.  That
>may be a thing which Linus wishes to do.  I'm easy.
>
>
>  
>



From: Andrew Morton [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Thu, 26 Aug 2004 01:45:42 -0700

Hans Reiser [email blocked] wrote:
>
>  >To get us started on this route it would really help me (and, probably,
>  >others) if you could describe what these API extensions are in a very
>  >simple way, without referring to incomprehehsible web pages,
>  >
>  what is not comprehensible....?

Pretty much anything at www.namesys.com.  The amount of time which is
needed to extract the technical info which one is looking for vastly
exceeds a gnat-like attention span.

As a starting point, please prepare a bullet-point list of
userspace-visible changes which the filesystem introduces, or is planned to
introduce.

And describe the "plugin" system.  Why does the filesystem need such a
thing (other filesystems get their features via `patch -p1')?

And what are the licensing implications of plugins?  Are they derived
works?  Must they be GPL'ed?



From: Hans Reiser [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Thu, 26 Aug 2004 02:24:41 -0700

Andrew Morton wrote:

>Hans Reiser [email blocked] wrote:
>  
>
>> >To get us started on this route it would really help me (and, probably,
>> >others) if you could describe what these API extensions are in a very
>> >simple way, without referring to incomprehehsible web pages,
>> >
>> what is not comprehensible....?
>>    
>>
>
>Pretty much anything at www.namesys.com.  The amount of time which is
>needed to extract the technical info which one is looking for vastly
>exceeds a gnat-like attention span.
>
>As a starting point, please prepare a bullet-point list of
>userspace-visible changes which the filesystem introduces, or is planned to
>introduce.
>
>And describe the "plugin" system.  Why does the filesystem need such a
>thing (other filesystems get their features via `patch -p1')?
>  
>
It takes 6 months or more to become competent to change a usual 
filesystem.  Creating a new reiser4 plugin is a weekend programmer fun 
hack to do.  Weekend programmers matter, because they tend to have 
clever ideas based on understanding a need they have.   How many people 
can easily add new features to ext3 or reiserfs V3?  Very few. 

What happens if you need a disk format change?

Well, in V4, you can easily compose a plugin from plugin methods of 
other plugins, write a little piece of code with the one thing you want 
different, and add it in.  Disk format changes, no big deal, add a new 
disk format plugin, or a new item plugin, or a new node plugin, etc., 
and you got your new format.

There is a huge difference between code that is designed for reuse, and 
code that is not.  That is the difference between V3 and V4.  We were 
looking at our V3 balancing code, and it special cased each different 
kind of item (an item is a piece of something, which the balancing code 
chops objects into as it squeezes them into nodes).  It looked like the 
complexity of the balancing code was going to be N squared, where N was 
the number of different kinds of items.

So, we created item handlers, and wrote balancing code that could slice 
and dice and merge any item that implemented all of the operations 
required of an item handler.  From there it grew, and we made everything 
pluggable.

Adding features to the new code is far less than the time cost of adding 
features to the old code.  I've seen Nikita complain that something 
would take 6 weeks to do (changing key assignment algorithms), and then 
it takes him 3 afternoons, and it was because of the plugins, because 
when we did something similar in V3 it took 3 man months.

>And what are the licensing implications of plugins?  Are they derived
>works?  
>
Yes.

Hans



From: Horst von Brand [email blocked]
Subject: Re: silent semantic changes with reiser4 
Date: 	Thu, 26 Aug 2004 14:12:32 -0400

Hans Reiser [email blocked] said:
> Andrew Morton wrote:

[...]

> > The fact that one filesystem will offer features which other
> > filesystems do not and cannot offer makes me queasy for some reason.

Not me. As long as it is clearly experimental stuff that can be removed at
the head hacker's whim, that is. 2.7 material.

> Andrew, we need to compete with WinFS and Dominic Giampaolo's filesystem 
> for Apple,

Says who?

>            and that means we need to put search engine and database 
> functionality into the filesystem.

Please don't. Unix works and is extremely popular because it _doesn't_ try
to be smart (policy vs mechanism distinction, simple abstractions that can
be combined endlessly). If you add this to the filesystems, you either redo
_all_ userland to understand _one_ particular way of doing "smart things"
(which will turn out to be almost exactly the dumbest possible for some
uses, and then you are stuck), or get lots of shards from broken apps (and
users, and sysadmins).

>                                     It takes 11 years of serious 
> research to build a clean storage layer able to handle doing that.  

Great! Build an experimental OS showing how to use it, and through that see
if the ideas hold any water _in real, day-to-day, down-to-earth, practice_.

> Reiser4 has done that, finally.  None of the other Linux filesystems 
> have.

How come nobody before you in the almost 30 years of Unix has ever seen the
need for this indispensable feature?

>        The next major release of ReiserFS is going to be bursting with 
> semantic enhancements, because the prerequisites for them are in place 
> now.  None of the other Linux filesystems have those prerequisites.  

To me that would mean that ReiserFS has no place in Linux.

> They won't be able to keep up with the semantic enhancements.  This 
> metafiles and file-directories stuff is actually fairly trivial stuff.

Maybe. But the question of it being of any use aren't trivially answered
"yes". My gut feeling is that the answer is a resounding "no", but that's
just me. Direcories are directories, if you want to ship directory-like
stuff around, use directories (or tarfiles, or whatever). Just look what
happened to the much, much easier stuff of splitting SUID/SGID into
capabilities: Nothing at all whatsoever, because they have no decent place
to stay in the hallowed Unix APIs. Sure, Linux is now large enough to be
able to _define_ APIs to follow, but even so it isn't at all easy to do so.

> Look guys, in 1993 I anticipated the battle would be here, and I build 
> the foundation for a defensive tower right at the spot MS and Apple are 
> now maneuvering towards.

Why place a protective tower around the pit into which they are trying
desperately to throw themselves? ;-)

>                           Help me get the next level on the tower before 
> they get here.  It is one hell of a foundation, they won't be able to 
> shake it, their trees are not as powerful.  Don't move reiser4 into vfs, 
> use reiser4 as the vfs.  Don't write filesystems, write file plugins and 
> disk format plugins and all the other kinds of plugins, and you won't be 
> missing any expressive power that you really want....

Better write libraries handling whatever weird format you have in mind for
each use. Even applications. I don't expect to be able to use emacs to edit
a JPEG or PostgreSQL database. Neither do I expect gimp to be able to edit
a poem. The current trend (which I heartily agree with) is to take stuff
_out_ of the kernel (for flexibility, access, development ease, even
performance).
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513



From: Markus Törnqvist [email blocked] 
Subject: Re: silent semantic changes with reiser4
Date: 	Fri, 27 Aug 2004 23:32:16 +0300

On Thu, Aug 26, 2004 at 02:12:32PM -0400, Horst von Brand wrote:

>> Andrew, we need to compete with WinFS and Dominic Giampaolo's filesystem 
>> for Apple,
>Says who?

People will say it when people stop using Linux on servers because
they can integrate metadata easier in other operating systems ;)

>Please don't. Unix works and is extremely popular because it _doesn't_ try
>to be smart (policy vs mechanism distinction, simple abstractions that can
>be combined endlessly). If you add this to the filesystems, you either redo

Just more moving little pieces to be combined endlessly.

I think it looks like Reiser4 semantics should go into VFS and not break
legacy posix compatibility.

If you don't want to use a filesystem like that, don't.
Maybe the reason Linux hasn't developed this yet is the fact that Linux
supports n+1 file systems and above and beyond that many userspace ones.
So there has been enough work in maintaining and competing the old
ones and not coming up with new ones.

>_all_ userland to understand _one_ particular way of doing "smart things"
>(which will turn out to be almost exactly the dumbest possible for some
>uses, and then you are stuck), or get lots of shards from broken apps (and
>users, and sysadmins).

This is a question if implementation.

What if the files-as-dirs have a ..metas/mainstream[1] file and are handled
by legacy applications as directories? Nothing breaks, but may look a bit
ugly. Nothing one line of shell scripting would not cure.

OTOH, the user space progs are quickly patched to support all this.

>Great! Build an experimental OS showing how to use it, and through that see
>if the ideas hold any water _in real, day-to-day, down-to-earth, practice_.

Why should he?

Parts of Linux are already here and susceptible to experiments in
evolution. (Break out 2.7 for this one, the people want code! :)

>How come nobody before you in the almost 30 years of Unix has ever seen the
>need for this indispensable feature?

Lack of imagination?
Lack of need?
Lack of drive space?

And maybe someone did think among these lines back in the days, it's
not like the current dogma of file systems is the only one that has
ever been used.

>To me that would mean that ReiserFS has no place in Linux.

This is the type of argumentation I can not understand.
Do you not have the freedom not to use it?

And this is not personal, but please say Reiser4, as ReiserFS is coupled
with versions 3.x. They are quite different.

>Maybe. But the question of it being of any use aren't trivially answered
>"yes". My gut feeling is that the answer is a resounding "no", but that's

I think it's a pretty big yes, even if the technical details do need to
be clear before anything is set in stone. That's the reason for the
on-going discussion.

>just me. Direcories are directories, if you want to ship directory-like
>stuff around, use directories (or tarfiles, or whatever). Just look what

Without a common user-space API to deal in them. Oh, joy.
Make the common user-space API? Will never happen.

>Better write libraries handling whatever weird format you have in mind for
>each use. Even applications. I don't expect to be able to use emacs to edit
>a JPEG or PostgreSQL database. Neither do I expect gimp to be able to edit
>a poem. The current trend (which I heartily agree with) is to take stuff
>_out_ of the kernel (for flexibility, access, development ease, even
>performance).

I don't think anyone's talking about using emacs to edit a picture, but
emacs could be used to edit the picture's description.[2]
Maybe with some userspace-hook that exports the exif data via ..metas/
so the kernel won't have to parse it.

Linus wrote about why TCP is not in userspace and I think this is a similar
situation. Technically. But not everything has to be in kernel space
in Reiser4. Or that's what they say and I'm not sure where I stand on
an exif parser in the kernel...

On the social side, I may be wrong in this, but not many userspace 
filesystems have the possibility of becoming a de-facto standard. One 
might say that's because they suck, or they suck because they're 
implemented in userspace or they are just met with prejudice
for the same reason.

In the kernel something like this would have to be done right and people
would put effort into doing it right.

I for one would welcome that.

[1] Sorry, guys, just had to use that name..
[2] You can edit a picture in emacs already, it's just prone to break ;)

-- 
mjt



From: Linus Torvalds [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Fri, 27 Aug 2004 13:38:28 -0700 (PDT)



On Fri, 27 Aug 2004, Markus Törnqvist wrote:
> 
> People will say it when people stop using Linux on servers because
> they can integrate metadata easier in other operating systems ;)

Heh. Considering that WinFS seems to be delayed yet more, I don't think 
that's a very strong argument.

Hell will freeze over before Microsoft does a filesystem right. Besides,
WinFS is likely almost in user mode anyway, ie mostly a library, rather
like the gnome people are already doing with nome storage.

So there's really no point in trying to push your agenda by trying to 
scare people with MS activities. Linux kernel developers do what's right 
because it is _right_, not because somebody else does it.

		Linus



From: Hans Reiser [email blocked]
Subject: Re: silent semantic changes with reiser4
Date: 	Sat, 28 Aug 2004 02:55:00 -0700

Linus Torvalds wrote:

>On Fri, 27 Aug 2004, Markus Törnqvist wrote:
>  
>
>>People will say it when people stop using Linux on servers because
>>they can integrate metadata easier in other operating systems ;)
>>    
>>
>
>Heh. Considering that WinFS seems to be delayed yet more, I don't think 
>that's a very strong argument.
>
>Hell will freeze over before Microsoft does a filesystem right. Besides,
>WinFS is likely almost in user mode anyway, ie mostly a library, rather
>like the gnome people are already doing with nome storage.
>
>So there's really no point in trying to push your agenda by trying to 
>scare people with MS activities. Linux kernel developers do what's right 
>because it is _right_, not because somebody else does it.
>
>		Linus
>
>
>  
>
Apple will get it right.  I promise it.  I have met Dominic, and he is 
very very sharp.  Look at the Tiger demos on their website.  Simple 
interface, looks nice to me.... 

The one area he might screw up is performance, but I don't care to count 
on that.

WinFS first tried to put it all in the FS, and then it became a user 
mode library almost certainly because they are making the standard 
mistakes the database guys make when they try to emulate file systems 
without changing the core balanced tree algorithms, and their 
performance sucked and they had to back off.  It took 11 years for me to 
get it right, and they aren't as crazy-err-persistent as I am.;-)

We might get lucky and have them produce another NTFS, but then again, 
when Microsoft focuses on a task, they do much better at it than they do 
most of the time, and they are focused on WinFS.  They have hired very 
sharp people.  We can hope that they don't know how to use them, but 
when they hire people like Gerard Salton for $1 million a year, there is 
just possibly a chance that they might try to get their money's worth 
out of him.

You should not be complacent about WinFS being delayed to 2007, because 
even if I get funding for enhanced ReiserFS semantics tomorrow we also 
can't get the job done before 2007.  This is big science, not writing a 
device driver.

Finally, how much harm will it be if we do it right and it is important 
and they fail?  Suppose I am wrong about them, and we create a powerful 
unifying namespace for Linux before any other OS does?  Is that so bad?

Creating a powerful namespace at the heart of Linux is the most 
important enhancement you can make to the OS. 

Finally the storage layer is good enough to support putting the 
relationship between keywords (actually keyobjects in my scheme....) and 
their documents directly into the FS without losing performance for 
traditional file system usage patterns, and I get to stop tweaking 
performance and go have fun with semantics in the next major release.

Hans

• Archive of above thread
• KernelTrap interview with Andrew Morton