Linux: Merging FUSE In 2.6.14

Submitted by Jeremy
on September 6, 2005 - 10:23am

In a recent thread, Miklos Szeredi provided some usage statistics for FUSE, the "Filesystem in Userspace". FUSE is an API that was originally developed for the AVFS project, and has since been utilized by a growing range of userland filesystems. It was merged into Andrew Morton's -mm kernel in mid January of 2005 [story], and nearly made it into the mainline 2.6.12 kernel in early March [story]. Miklos pointed out that the code has been downloaded around 7,000 times in the past three months, is successfully utilized by 27 known userland filesystems, and provides bindings in 7 different programming languages.

2.6 Linux kernel maintainer Andrew Morton replied, "the main sticking point with FUSE remains the permission tricks around fuse_allow_task(). AFAIK it remains the case that nobody has come up with any better idea, so I'm inclined to merge the thing." He went on to note, "what others think matters. But at this stage, objections would need to be substantial, IMO. We're rather deadlocked on the permission thing, but if we can't come up with anything better then I'm inclined to say what-the-hell." In a later email [story], Andrew had made up his mind noting that he was, "fed up with arguing - any remaining problems can be fixed up in-tree if anyone can think of how to fix them."


From: Miklos Szeredi [email blocked]
To:  akpm
Subject: FUSE merging?
Date:	Sat, 03 Sep 2005 00:02:18 +0200

Hi Andrew!

Do you plan to send FUSE to Linus for 2.6.14?

I know you have some doubts about usefulness, etc.  Here are a couple
of facts, that I hope show that Linux should benefit from having FUSE:

 - total number of downloads from SF: ~25000

 - number of downloads of last release (during 3 months): ~7000

 - number of distros carrying official packages: 2 (debian, gentoo)

 - number of publicly available filesystems known: 27

 - of which at least 2 are carried by debian (and maybe others)

 - number of language bindings: 7 (native: C, java, python, perl, C#, sh, TCL)

 - biggest known commercial user: ~110TB exported, total bandwidth: 1.5TB/s

 - mailing list traffic 100-200 messages/month

 - have been in -mm since 2005 january

Miklos


From: Andrew Morton [email blocked] Subject: Re: FUSE merging? Date: Fri, 2 Sep 2005 15:34:40 -0700 Miklos Szeredi [email blocked] wrote: > > Hi Andrew! > > Do you plan to send FUSE to Linus for 2.6.14? Haven't thought about it all much. Have spent most of my time in the last month admiring the contents of kernel bugzilla, and the ongoing attempts to increase them. > I know you have some doubts about usefulness, etc. Here are a couple > of facts, that I hope show that Linux should benefit from having FUSE: > > - total number of downloads from SF: ~25000 > > - number of downloads of last release (during 3 months): ~7000 > > - number of distros carrying official packages: 2 (debian, gentoo) > > - number of publicly available filesystems known: 27 > > - of which at least 2 are carried by debian (and maybe others) > > - number of language bindings: 7 (native: C, java, python, perl, C#, sh, TCL) > > - biggest known commercial user: ~110TB exported, total bandwidth: 1.5TB/s > > - mailing list traffic 100-200 messages/month > > - have been in -mm since 2005 january > I agree that lots of people would like the functionality. I regret that although it appears that v9fs could provide it, there seems to be no interest in working on that. The main sticking point with FUSE remains the permission tricks around fuse_allow_task(). AFAIK it remains the case that nobody has come up with any better idea, so I'm inclined to merge the thing.
From: Miklos Szeredi [email blocked] Subject: Re: FUSE merging? Date: Sat, 03 Sep 2005 07:31:34 +0200 > Haven't thought about it all much. Have spent most of my time in the last > month admiring the contents of kernel bugzilla, and the ongoing attempts to > increase them. A penal system could be created, for example if someone is caught introducing a bug, he will have to choose three additional reports from bugzilla and analyze/fix them ;) > > - number of language bindings: 7 (native: C, java, python, perl, > > - C#, sh, TCL) 8 now, someone just sent a private mail about bindings for the Pliant (never heard of it) language. > I agree that lots of people would like the functionality. I regret that > although it appears that v9fs could provide it, I think you are wrong there. You don't appreciate all the complexity FUSE _lacks_ by not being network transparent. Just look at the error text to errno conversion muck that v9fs has. And their problems with trying to do generic uid/gid mappings. > there seems to be no interest in working on that. It would mean adding a plethora of extensions to the 9P protocol, that would take away all it's beauty. I think you should realize that these are different interfaces for different purposes. There may be some overlap, but not enough to warrant trying to massage them into one big ball. > The main sticking point with FUSE remains the permission tricks around > fuse_allow_task(). AFAIK it remains the case that nobody has come up with > any better idea, so I'm inclined to merge the thing. Do you promise? I can do a resplit and submit to Linus, if that takes some load off you. Thanks, Miklos
From: Andrew Morton [email blocked] Subject: Re: FUSE merging? Date: Fri, 2 Sep 2005 23:40:42 -0700 Miklos Szeredi [email blocked] wrote: > > > The main sticking point with FUSE remains the permission tricks around > > fuse_allow_task(). AFAIK it remains the case that nobody has come up with > > any better idea, so I'm inclined to merge the thing. > > Do you promise? I troll. What others think matters. But at this stage, objections would need to be substantial, IMO. We're rather deadlocked on the permission thing, but if we can't come up with anything better then I'm inclined to say what-the-hell. > I can do a resplit and submit to Linus, if that takes > some load off you. Nah, then I'd just have to check that everything is the same.
From: Miklos Szeredi [email blocked] Subject: Re: FUSE merging? Date: Sat, 03 Sep 2005 09:23:31 +0200 > > > The main sticking point with FUSE remains the permission tricks around > > > fuse_allow_task(). AFAIK it remains the case that nobody has come up with > > > any better idea, so I'm inclined to merge the thing. > > > > Do you promise? > > I troll. What others think matters. But at this stage, objections would > need to be substantial, IMO. Fair enough. > We're rather deadlocked on the permission thing, but if we can't > come up with anything better then I'm inclined to say what-the-hell. There's no disadvantage IMO. It adds nearly zero complexity. If someone doesn't like it, it can be configured out in userspace. And it leaves no legacy interfaces to support if later a better method is found. > > I can do a resplit and submit to Linus, if that takes > > some load off you. > > Nah, then I'd just have to check that everything is the same. OK. Thanks, Miklos

Related Links:

Amen!

on
September 6, 2005 - 1:42pm

This would be a welcomed addition. Some of the filesystems enabled by FUSE are absolutely wonderful--encfs, GfarmFS.

New ideas for file systems...

on
September 6, 2005 - 4:59pm

Here's a list of potential uses for userspace file systems. I wonder what else could be dreamed up.

kio-fuse

asdex (not verified)
on
September 7, 2005 - 4:49pm

kio-fuse already exists. It enables you to mount all the KDE io-slaves (ftp, ssh, sftp, webdav, ...)

kio-fuse backwards?

on
September 8, 2005 - 6:34am

How about an implementation the other way around? Allow kio to use existing fuse file systems even on systems where the administrator has decided not to install the fuse kernel module.

Re: kio-fuse backwards

Vlad C. (not verified)
on
September 9, 2005 - 5:13pm

My thoughts exactly! However, there is _staunch_ opposition (bordering on NIH syndrome) from KDE developers who are only satisfied with KDE-centric network protocol implementations, which IMHO are suboptimal compared to FUSE for the following reasons:

1) KDE-based implementations only work on KDE apps, thus denying shell utilities, GTK, Motif (in essence everything else) access to network locations. Allowing non-KDE apps to use KDE-based network framework takes a lot of work on the part of the app developers (like embedding miniature clients for every supported protocol), something not even Mozilla Firefox and OpenOffice have been able to muster with their relatively large developer base.

2) KDE-based implementations can’t stream data, cutting off most multimedia uses.

Check out this thread for a detailed discussion of user-friendly network access:

Re: kio-fuse backwards

Anonymous (not verified)
on
September 10, 2005 - 5:42pm

1) The aforementioned kio-fuse allows non-KDE programs to use KIO.
2) I don't think this is true. KHTML can do progressive rendering of HTML pages, for example. Also, I've used Noatun to play Ogg Vorbis streams over HTTP. (It is true that most KParts must wait until the entire file is received before displaying them, maybe that's what you're thinking of?)

Re: kio-fuse backwards

Vlad C. (not verified)
on
September 10, 2005 - 10:01pm

1) KIO-FUSE may be a solution, but I would expect there to be a performance penalty for going through the KDE layer. It would be interesting to see a benchmark comparing SSHFS versus KIO-SSH and SMB for FUSE versus KIO-SMB.

2) I've tried opening MP3 files via KIO-SSH and KIO-SMB in Kaffeine, and it gave an error saying “Remote files not accepted. You can only select local files.” The same files streamed perfectly in Kaffeine using SSHFS and SMB for FUSE.

Apparently I'm not the only one experiencing these difficulties with network KIO slaves. The developer of SMB for FUSE states on the project's website that:

“KIO Slaves cannot stream files making movie/music playing over LAN impossible.”

Re: kio-fuse backwards

Anonymous (not verified)
on
September 11, 2005 - 6:45pm

2) I seem to remember getting a similar message from JuK. My guess is that these programs check which KIO slave the user specified and refuse to use any other than "file". (Probably this is because they are not designed for streaming, e.g. they can not handle network latency like XMMS can.) It would be interesting to try doing the same thing by way of KIO-FUSE.

However, both amaroK and Noatun can stream audio over (at least) HTTP, and it would surprise me if they don't use KIO HTTP. (Noatun would usually get stuck after playing about half a minute of audio, but I think that because of the aformentioned latency problem combined with the fact that I was trying to stream over a PPP connection. amaroK didn't have this problem.)

2) Maybe Kaffeine is using Xi

Anonymous Coward (not verified)
on
December 4, 2005 - 1:51am

2) Maybe Kaffeine is using Xine or some other non-KDE engine? So it may be able to play over some common protocols like HTTP if that engine supports it, but definitely not SSH or SMB.

NTFS

billgates99 (not verified)
on
September 7, 2005 - 3:10pm

Does this mean that I can finally get a NTFS fuser module, and never have to keep compiling ntfs for each new kernel release?

re: NTFS FUSE

FlatCap (not verified)
on
September 7, 2005 - 5:17pm

We have a new program "ntfsmount" that uses the FUSE module. It can create / delete files and directories. It works, but it hasn't been heavily tested, yet. Beware.

FlatCap (Rich)

re: NTFS FUSE

Yura Pakhuchiy (not verified)
on
September 11, 2005 - 4:06am

File creation and deletion is implemented only in CVS version of ntfsmount. Latest release 1.11.2 can only write to files *with* changes to file size. Visit linux-ntfs.org and wiki.linux-ntfs.org for more information.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.