In the debate following Andrew Morton [interview [1]] posting his plans for 2.6.13 [story [2]], the existence of a plugin layer [3] in Reiser4 [4] was discussed. Jeff Garzik put it blunty, "the plugin stuff is crap. This is not a filesystem but a filesystem new layer. IMO considered in that light, it duplicates functionality elsewhere." Andrew Morton went on to explain, "I think the concern here is that this is implemented at the wrong level. In Linux, a filesystem is some dumb thing which implements address_space_operations, filesystem_operations, etc."
Hans Reiser noted, "please remember that this is per file, per item, per node, per attribute, per disk format, per bitmap, per super block, etc., abstracting, not per filesystem abstracting." He explained a couple advantages to plugins being that it makes it much easier for developers to change the disk format, and allows for easy code reuse. He added, "the use of plugins forced all the programmers to think about reusability at every layer of design. V3 of reiserfs is way too hard to work on and modify. If you ask one of the team to code something for V3 instead of V4, they quietly groan at the thought. It is just so much easier to do in V4."
Andrew Morton replied, "advanced features such as those which you describe are implemented on top of the filesystem, not within it. reiser4 turns it all upside down. Now, some of the features which you envision are not amenable to above-the-fs implementations. But some will be, and that's where we should implement those." The lengthy discussion [5] continued, an interesting read for Reiser4 supporters and detractors alike.
From: Hans Reiser [email blocked] To: Jeff Garzik [email blocked] Subject: Re: -mm -> 2.6.13 merge status Date: Tue, 21 Jun 2005 13:05:44 -0700 Jeff Garzik wrote: > > >> reiser4 >> >> > > > The plugin stuff is crap. This is not a filesystem but a filesystem + > new layer. IMO considered in that light, it duplicates functionality > elsewhere. What functionality where? Please remember that this is per file, per item, per node, per attribute, per disk format, per bitmap, per super block, etc., abstracting, not per filesystem abstracting. Plugins allow a number of things: 1) They allow us to never pay the cost to change the disk format again, no matter how much we add in future years. This really matters: the prohibitive cost of disk format changes are the number one impediment to filesystem improvements, and the primary reason why most filesystems ossify after time has past. 2) They allow us to easily structure code for reuse. If we want to create a new kind of file that is like some other kind of file except for one thing, we just write the one thing, and then easily reuse all the other code, and create a new plugin id. The use of plugins forced all the programmers to think about reusability at every layer of design. V3 of reiserfs is way too hard to work on and modify. If you ask one of the team to code something for V3 instead of V4, they quietly groan at the thought. It is just so much easier to do in V4. When I asked one of our team to completely change the key assignment algorithm for V4 (which controls what things get packed near what in the tree), he complained that it would take 6 weeks to do it. Under V3 it would have taken 3 months. It took him 3 days, and now to change it again would take him 3 hours I think. Oh, by the way, the change boosted performance dramatically. If you take the time to become familiar with coding within our plugin layer, I think you will find yourself wanting the same to exist for other filesystems. Of course, no other filesystem needs to be impacted by our plugin layer, and there is no way reiser4 could easily be rewritten to exist without it (it would be like requiring that the kernel not use function calls and only use gotos). Reiser4's plugin layer has as its explicit objective making it possible for the weekend hacker to add something useful to reiser4 and send it in for inclusion. We want to democratize filesystem innovation so that random bright guys who usually work on something other than filesystems can act on their bright ideas without spending 3 years in the filesystem field to do it. I believe that most great filesystem innovations are envisioned by persons not working on filesystems, and go nowhere because of the especially high cost of entry into our field. I am not as bright as other filesystem developers, and so we must tinker with three ideas and keep one of them. The speed of tinkering is crucial to us, and the plugin layer increases that speed several fold. Please reconsider your view.
From: Christoph Hellwig [email blocked] Subject: Re: -mm -> 2.6.13 merge status Date: Tue, 21 Jun 2005 21:24:48 +0100 Hans, we had this discussion before. And the consensus was pretty simple: the disk structure plugins are your business and fine to keep. The higher-level pluging that just add another useless abstraction below file_operation/inode_operation/etc. are not. keep the former and kill the latter and you're one step further.
From: Hans Reiser [email blocked] Subject: reiser4 plugins Date: Tue, 21 Jun 2005 18:07:58 -0700 Christoph, Reiser4 users love the plugin concept, and all audiences which have listened to a presentation on plugins have been quite positive about it. Many users think it is the best thing about reiser4. Can you articulate why you are opposed to plugins in more detail? Perhaps you are simply not as familiar with it as the audiences I have presented to. Perhaps persons on our mailing list can comment..... In particular, what is wrong with having a plugin id associated with every file, storing the pluginid on disk in permanent storage in the stat data, and having that plugin id define the set of methods that implement the vfs operations associated with a particular file, rather than defining VFS methods only at filesystem granularity? What is wrong with having an encryption plugin implemented in this manner? What is wrong with being able to have some files implemented using a compression plugin, and others in the same filesystem not. What is wrong with having one file in the FS use a write only plugin, in which the encrypion key is changed with every append in a forward but not backward computable manner, and in order to read a file you must either have a key that is stored on another computer or be reading what was written after the moment of cracking root? What is wrong with having a set of critical data files use a CRC checking file plugin? What we have hurts no one but us. I have never seen an audience for one of my talks that thought it hurt us..... most audiences think it is great. Let us tinker with our FS, and you tinker with yours, and so long as what we do does not affect your FS, let the users choose. In the end, somebody will write a new fs that steals the good ideas from both of us, and obsoletes us both. They can only do this though if we are left to be both free to implement differing filesystem designs. I do not tell you how to design XFS, why are you making my life unpleasant? Christoph Hellwig wrote: >Hans, we had this discussion before. And the consensus was pretty simple: >the disk structure plugins are your business and fine to keep. The >higher-level pluging that just add another useless abstraction below >file_operation/inode_operation/etc. are not. keep the former and kill >the latter and you're one step further. > > > >
From: Jeff Garzik [email blocked] Subject: Re: reiser4 plugins Date: Tue, 21 Jun 2005 21:14:06 -0400 Hans Reiser wrote: > Christoph, > > Reiser4 users love the plugin concept, and all audiences which have > listened to a presentation on plugins have been quite positive about > it. Many users think it is the best thing about reiser4. Can you > articulate why you are opposed to plugins in more detail? Perhaps you > are simply not as familiar with it as the audiences I have presented > to. Perhaps persons on our mailing list can comment..... > > In particular, what is wrong with having a plugin id associated with > every file, storing the pluginid on disk in permanent storage in the > stat data, and having that plugin id define the set of methods that > implement the vfs operations associated with a particular file, rather > than defining VFS methods only at filesystem granularity? You're basically implementing another VFS layer inside of reiser4, which is a big layering violation. This sort of feature should -not- be done at the low-level filesystem level. What happens if people decide plugins are a good idea, and they want them in ext3? We need massive surgery to extract the guts from reiser4. Jeff
From: David Masover [email blocked] Subject: Re: reiser4 plugins Date: Tue, 21 Jun 2005 23:25:24 -0500 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeff Garzik wrote: > Hans Reiser wrote: > >> Christoph, >> >> Reiser4 users love the plugin concept, and all audiences which have >> listened to a presentation on plugins have been quite positive about >> it. Many users think it is the best thing about reiser4. Can you >> articulate why you are opposed to plugins in more detail? Perhaps you >> are simply not as familiar with it as the audiences I have presented >> to. Perhaps persons on our mailing list can comment..... >> >> In particular, what is wrong with having a plugin id associated with >> every file, storing the pluginid on disk in permanent storage in the >> stat data, and having that plugin id define the set of methods that >> implement the vfs operations associated with a particular file, rather >> than defining VFS methods only at filesystem granularity? > > > You're basically implementing another VFS layer inside of reiser4, which > is a big layering violation. There's been sloppy code in the kernel before. I remember one bit in particular which was commented "Fuck me gently with a chainsaw." If I remember correctly, this had all of the PCI ids and the names and manufacturers of the corresponding devices -- in a data structure -- in C source code. It was something like one massive array definition, or maybe it was a structure. I don't remember exactly, but... The point is, this was in the kernel for quite awhile, and it was so ugly that someone would rather be fucked with a chainsaw. If something that bad can make it in the kernel and stay for awhile because it worked, and no one wanted to replace it -- nowdays, that database is kept in userland as some nice text format -- then I vote for putting Reiser4 in the kernel now, and ironing the sloppiness ("violation") out later. It runs now. > This sort of feature should -not- be done at the low-level filesystem > level. I agree there, too. In fact, some people have suggested that all "legacy" (read: non-reiser) filesystems should be implemented as Reiser4 plugins, effectively killing VFS.* So, Reiser4 may eventually take over VFS and be the only Linux filesystem, but if so, it will have to do it much more slowly. Take the good ideas -- things like plugins -- and make them at least look like incremental updates to the current VFS, and make them available to all filesystems. Eventually, this would result in a full merge of Reiser and Linux, such that the only thing left of "Reiser4" are a few plugins -- things like storage plugins and maybe some more exotic things like fibration that I don't really understand. > What happens if people decide plugins are a good idea, and they want > them in ext3? We need massive surgery to extract the guts from reiser4. And here is the crucial point. Reiser4 is usable and useful NOW, not after it has undergone massive surgery, and Namesys is bankrupt, and users have given up and moved on to XFS. But the massive surgery should happen eventually, partly to make all filesystems better (see below), and partly to make the transition easier and more palatable for those who don't work for Namesys. * Imagine ext3 as a storage-level plugin for reiser4. You get one benefit immediately -- lazy allocation. Lazy allocation is nice both for fragmentation and for busy systems writing and nuking a lot of temporary files. Imagine a file which is written and then deleted before it hits disk -- it should never touch the disk, regardless of the underlying storage layout. Other benefits are subtler but inevitable. Right now, it would be an understatement to say that there's duplication of effort between ext3, xfs, and reiser4. And yet, I don't think there are many core design decisions that influence my decision as to which I pick up. I just want it to be fast, stable, and have a stable on-disk format so I don't have to reformat too often. I honestly don't care about how XFS is segmenting the disk, or Reiser4 packs really well, or ext3 can be read as ext2 and flushes to disk every five seconds. These are the kinds of things which should be set to sane defaults, tunable for enterprise users, but are not really enough to warrant completely separate code bases. I would imagine that it wouldn't be too long after this before an uber-fs rose, something which combined enough of the strong points of all the existing Linux filesystems that no one would use anything else. But, Linux still needs support for FAT32, ISO9660, UDF, and all the other filesystems which won't go away as easily as XFS and ext3, and it would be nice if these could all share as much code as possible. I don't know if storage plugins really work that way, but they should. I think. I don't work here. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org [6] iQIVAwUBQrjoNHgHNmZLgCUhAQIYYw/7BWZ0gVvy0ln5tRo405yUoRHJ/jVFBGyP 5pQ610ECMZORVWRO1qP/NXbvGZwDjEggM5iIeahsGqnBWzNGDsB6RslMUZAxzCYy iAovi0881zoARf3dmhKrDgbGkvNLPTx+/ypa20oRcHLnyI+NAjerUxNuvGc79PJn Ybm9JhX6tQsqGIKjZye9uNDHCifLqzA1gdxucPwWo9sz4ymzM9FgsMGvb+IxrU50 2a2G2K6AHcH+nkomEHw3xgY3PmUZUy0s2hQDSkJx6dCido7fGZwwykaSXm4IZs9s xZqBxKx32G/LDnDV3W8gpj8ZisUE+58kefRbIjo4Ml6IzgXvQqpRjaQOuz8JoKDX 9KUV43tcZkPpK+neIYPQYpXCrdMQqB5+ISpbNKuz/Z/abkcVw1042sy0EKM+/VnD n3Jr7PpSyk0lfCyADr+33qnqPQRFq02gQTohg9FheoMthhV01aaeGW5ExmTM/Wwa 6Dmv/qnn2oNImi+Ebz5u3Lbnz7lL3MVpRL4aoMmEOVUB3xhehnxesf//yacBmwj9 M/3KVae9epwX4K6yi8qQXzH4160IBFHpWUxBLc9LnOZlHQuZ+Fz3BIrbKQBvmaHT 7lrwi0Os6TmiBGMSFd6OUWHcYs4p97aMw30NG33fCL6e8X6oNVFwwnJXZpBPN1Mr +lrsVAywKEI= =rHdK -----END PGP SIGNATURE-----
From: Jeff Garzik [email blocked] Subject: Re: reiser4 plugins Date: Wed, 22 Jun 2005 01:18:52 -0400 David Masover wrote: > There's been sloppy code in the kernel before. I remember one bit in > particular which was commented "Fuck me gently with a chainsaw." If I > remember correctly, this had all of the PCI ids and the names and > manufacturers of the corresponding devices -- in a data structure -- in > C source code. It was something like one massive array definition, or > maybe it was a structure. I don't remember exactly, but... > > The point is, this was in the kernel for quite awhile, and it was so > ugly that someone would rather be fucked with a chainsaw. If something > that bad can make it in the kernel and stay for awhile because it > worked, and no one wanted to replace it -- nowdays, that database is > kept in userland as some nice text format -- then I vote for putting > Reiser4 in the kernel now, and ironing the sloppiness ("violation") out > later. It runs now. Existence of ugly code is not an excuse to add more. We have to maintain said ugly code for decades. Maintainability is a big deal when you deal with the timeframes we deal with. > So, Reiser4 may eventually take over VFS and be the only Linux > filesystem, but if so, it will have to do it much more slowly. Take the > good ideas -- things like plugins -- and make them at least look like > incremental updates to the current VFS, and make them available to all > filesystems. This is how all Linux development is done. The code evolves over time. You have just described the path ReiserFS needs to take, in order to get this stuff into the kernel, in fact. > And here is the crucial point. Reiser4 is usable and useful NOW, not > after it has undergone massive surgery, and Namesys is bankrupt, and > users have given up and moved on to XFS. But the massive surgery should > happen eventually, partly to make all filesystems better (see below), > and partly to make the transition easier and more palatable for those > who don't work for Namesys. We care about technical merit, not some random company's financial solvancy. Reiser4 has layering violations, and doesn't work with the current security layer. Those are two biggies. There is no entitlement to be merged in the upstream kernel. If people don't like how the Linux kernel is managed, they are free to maintain their own fork. If its better, people will use that instead. > I would imagine that it wouldn't be too long after this before an > uber-fs rose, something which combined enough of the strong points of > all the existing Linux filesystems that no one would use anything else. > But, Linux still needs support for FAT32, ISO9660, UDF, and all the > other filesystems which won't go away as easily as XFS and ext3, and it > would be nice if these could all share as much code as possible. > > > I don't know if storage plugins really work that way, but they should. No, they shouldn't. > I think. I don't work here. Obviously. Jeff
From: Andrew Morton [7] [email blocked] Subject: Re: reiser4 plugins Date: Tue, 21 Jun 2005 18:18:02 -0700 Hans Reiser [email blocked] wrote: > > What is wrong with having an encryption plugin implemented in this > manner? What is wrong with being able to have some files implemented > using a compression plugin, and others in the same filesystem not. > > What is wrong with having one file in the FS use a write only plugin, in > which the encrypion key is changed with every append in a forward but > not backward computable manner, and in order to read a file you must > either have a key that is stored on another computer or be reading what > was written after the moment of cracking root? > > What is wrong with having a set of critical data files use a CRC > checking file plugin? I think the concern here is that this is implemented at the wrong level. In Linux, a filesystem is some dumb thing which implements address_space_operations, filesystem_operations, etc. Advanced features such as those which you describe are implemented on top of the filesystem, not within it. reiser4 turns it all upside down. Now, some of the features which you envision are not amenable to above-the-fs implementations. But some will be, and that's where we should implement those.
From: Andi Kleen [email blocked] Subject: Re: reiser4 plugins Date: 22 Jun 2005 03:26:26 +0200 First Hans let me remind you that this discussion is not XFS vs reiser4. Christoph does a lot of reviewing and your child definitely is in serious need of that to be mergeable. I'm sure Christoph is able to review inpartially even when he is involved with other FS. Jeff Garzik [email blocked] writes: > > You're basically implementing another VFS layer inside of reiser4, > which is a big layering violation. > > This sort of feature should -not- be done at the low-level filesystem level. > > What happens if people decide plugins are a good idea, and they want > them in ext3? We need massive surgery to extract the guts from > reiser4. We already kind of have them, there are toolkits to do stackable FS with the existing VFS. However I suspect Hans is unwilling to redo his file system in this point. While it looks quite unnecessary there might be other areas which deserve more attention. In general all the code needs review, even if it is not as controversal as the reinvented VFS. -Andi
From: Hans Reiser [email blocked] Subject: Re: reiser4 plugins Date: Tue, 21 Jun 2005 19:47:13 -0700 Andi Kleen wrote: > Christoph does a lot of reviewing > and he is notorious for making needed linux contributors go away and not come back, and I won't say which famous person on this mailing list told me that.... >and your child definitely >is in serious need of that to be mergeable. I'm sure Christoph is able >to review inpartially even when he is involved with other FS. > > As impartial as a puppy on PCP.... Christoph is aggressive about things he does not take the time to understand or ask about first. I hate that. I wish he would go away please. He is not exactly an Ousterhout, Rob Pike, Granger, Mazieres, Frans Kaashoek, etc., in his accomplishments, so why is he reviewing other people's filesystems? Reviews are great, how about finding persons who have created filesystem innovations (and thus are less likely to reject innovations without understanding them) to do them? How about review by benchmark instead? It works, it runs faster than the competition, users like it, we addressed the core kernel patch complaints, it should go in and receive the exposure that will result in lots of useful improvements and suggestions. It seems like we are getting an unusual review process. I used a bunch of algorithms for which there was a consensus among those consulted that the algorithms were a bad idea, only the fact that I paid for the code to be written and required that it be done my way (ignoring the "you just use me as a typewriter" remarks as best I could) caused the algorithms to get implemented at all, and the benchmarks then proved the algorithms were a good idea. V3 performance sucks in major part because I listened to the consensus when I knew better. I don't really care for consensus on my FS anymore. If you guys want to understand what I am doing I am happy to talk about it extensively, but please don't require that I groupthink. I frankly think that with my benchmarks, I should be allowed to tinker on my own. Hans The Mad
From: Kyle Moffett [email blocked] Subject: Re: reiser4 plugins Date: Tue, 21 Jun 2005 23:16:31 -0400 On Jun 21, 2005, at 22:47:13, Hans Reiser wrote: > Andi Kleen wrote: >> and your child definitely >> is in serious need of that to be mergeable. I'm sure Christoph is >> able >> to review inpartially even when he is involved with other FS. > As impartial as a puppy on PCP.... So where else are you planning to get a competent reviewer who is fluent in the internals of filesystems? Good reviewers don't grow on trees, and in order to be able to understand filesystem issues, one must generally have worked with them before... Besides, what good is insulting others going to do? > Christoph is aggressive about things he does not take the time to > understand or ask about first. [rant snipped] From my objective re-reading of his posts, I can see that he is critical of things that are difficult to understand not just to be critical, but to provoke additional thought over those portions of the code. In many cases this leads to better abstractions and simpler code than otherwise. > How about review by benchmark instead? /dev/sda is a great filesystem with awesome benchmarks, assuming one only needs to store a single file. Besides, benchmarks aren't the only thing important about code. If the interface consists of: void clear_current_filename(void); void add_char_to_current_filename(char x); void read_bytes_from_current_file(char *byte, unsigned long size); void write_bytes_to_current_file(const char *byte, unsigned long size); then this is clearly not a good API, regardless of how well or poorly it may perform. > It works, it runs faster than the competition, users like it, we > addressed the core kernel patch complaints, it should go in and > receive > the exposure that will result in lots of useful improvements and > suggestions. It seems like we are getting an unusual review process. If you look over other patches in -mm, you will see that your review process is not unusual, especially given the number of concerns that other developers have raised over Reiser4. [irrelevant algorithm rant snipped] > If you guys want to understand > what I am doing I am happy to talk about it extensively, but please > don't require that I groupthink. I frankly think that with my > benchmarks, I should be allowed to tinker on my own. Yes, you can tinker on your own all you want. Another project that has taken that route is GrSecurity, which was alive and well last I checked. If you don't like others criticisms, take up your marbles and go home, just don't expect them to accept your work when you've not fixed it to community standards. Cheers, Kyle Moffett -- Somone asked my why I work on this free (http://www.fsf.org/philosophy/ [8]) software stuff and not get a real job. Charles Shultz had the best answer: "Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life." -- Charles Shultz
Related Links:
- Archive of above thread [9]
- KernelTrap interview with Andrew Morton [10]