login
Search
A recent discussion and series of patches on the Linux Kernel mailing list looked at removing relayfs [story], moving its functionality into an API using sysfs. Paul Mundt offered the patches summarizing, "this is a small patch set for getting rid of relayfs, and moving the core of its functionality to kernel/relay.c. The API is kept consistent for everything but the relayfs-specific bits, meaning people will have to use other file systems to implement relay channel buffers." The thread offers more explanation as to the benefits of the change, with quite a number of people supporting the patches.
Relayfs is described as "a bunch of per-cpu kernel buffers that can be efficiently written into from kernel code. These buffers are represented as files which can be mmap'ed and directly read from in user space. The purpose of this setup is to provide the simplest possible mechanism allowing potentially large amounts of data to be logged in the kernel and 'relayed' to user space." It was originally merged into Andrew Morton [interview]'s -mm tree in January of 2005 [story], and finally made it into the mainline kernel with the release of 2.6.14.
From: Paul Mundt [email blocked] To: zanussi, Patrick Mochel [email blocked] Subject: [PATCH, RFC] sysfs: relay channel buffers as sysfs attributes Date: Sun, 19 Feb 2006 19:17:48 +0200 Now with relayfs integrated and the relay_file_operations exported for use by other file systems, I wonder what people think about adding in a sysfs attribute for setting up channel buffers. The conventional relayfs doesn't make a lot of sense for the use cases where there are multiple devices to stream data from, particularly if they're already mapped out through the driver model. Rather than duplicating device enumeration, simply adding this as an attribute seems to work reasonably well. Tom did some work on the rchan_callbacks for more easily implementing relay files in other file systems, and it would be nice to use this in a non-debug context, without duplicating device enumeration in multiple locations.
From: Christoph Hellwig [email blocked] Subject: Re: [PATCH, RFC] sysfs: relay channel buffers as sysfs attributes Date: Sun, 19 Feb 2006 17:21:20 +0000 Hmm, this actually makes a lot of sense. At this point we should probably move the guts of relayfs into kernel/relay.c or something and add a different config option for it. Given this and debugs we could probably kill the separate relayfs filesystem implementation.
From: Greg KH [email blocked] Subject: Re: [PATCH, RFC] sysfs: relay channel buffers as sysfs attributes Date: Sun, 19 Feb 2006 09:56:23 -0800 Note that Pat isn't the sysfs maintainer anymore :) On Sun, Feb 19, 2006 at 07:17:48PM +0200, Paul Mundt wrote: > Now with relayfs integrated and the relay_file_operations exported for > use by other file systems, I wonder what people think about adding in a > sysfs attribute for setting up channel buffers. > > The conventional relayfs doesn't make a lot of sense for the use cases > where there are multiple devices to stream data from, particularly if > they're already mapped out through the driver model. Rather than > duplicating device enumeration, simply adding this as an attribute seems > to work reasonably well. > > Tom did some work on the rchan_callbacks for more easily implementing > relay files in other file systems, and it would be nice to use this in a > non-debug context, without duplicating device enumeration in multiple > locations. Looks good, I like it. This properly handles the module owner stuff, too, right? And I agree with Christoph, with this change, you don't need a separate relayfs mount anymore. thanks, greg k-h
From: Paul Mundt [email blocked] Subject: Re: [PATCH, RFC] sysfs: relay channel buffers as sysfs attributes Date: Sun, 19 Feb 2006 20:52:54 +0200 On Sun, Feb 19, 2006 at 09:56:23AM -0800, Greg KH wrote: > Note that Pat isn't the sysfs maintainer anymore :) > My mistake, I'll check MAINTAINERS instead of the file comments next time. > On Sun, Feb 19, 2006 at 07:17:48PM +0200, Paul Mundt wrote: > > Now with relayfs integrated and the relay_file_operations exported for > > use by other file systems, I wonder what people think about adding in a > > sysfs attribute for setting up channel buffers. > > Looks good, I like it. This properly handles the module owner stuff, > too, right? > Could you elaborate on which module owner issue you're referring to? struct relay_attribute encapsulates a struct attribute, and it's handled the same way as the other attribute types (I modelled it after struct bin_attribute), and I don't see any places that I missed. When setting up the relay attribute, it's just a matter of: static struct relay_attribute dev_relay_attr = { .attr = { .owner = THIS_MODULE, ... }, ... }; Let me know if I've missed anything. > And I agree with Christoph, with this change, you don't need a separate > relayfs mount anymore. > Yes, that's where I was going with this, but I figured I'd give the relayfs people a chance to object to it going away first. If with this in sysfs and simple handling through debugfs people are content with the relay interface for whatever need, then getting rid of relayfs entirely is certainly the best option. We certainly don't need more pointless virtual file systems. I'll work up a patch set for doing this as per Cristoph's kernel/relay.c suggestion. Thanks for the feedback.
From: Tom Zanussi [email blocked] Subject: Re: [PATCH, RFC] sysfs: relay channel buffers as sysfs attributes Date: Sun, 19 Feb 2006 23:29:23 -0600 Paul Mundt writes: > On Sun, Feb 19, 2006 at 09:56:23AM -0800, Greg KH wrote: > [...] > > And I agree with Christoph, with this change, you don't need a separate > > relayfs mount anymore. > > > Yes, that's where I was going with this, but I figured I'd give the > relayfs people a chance to object to it going away first. > > If with this in sysfs and simple handling through debugfs people are > content with the relay interface for whatever need, then getting rid of > relayfs entirely is certainly the best option. We certainly don't need > more pointless virtual file systems. > > I'll work up a patch set for doing this as per Cristoph's kernel/relay.c > suggestion. Thanks for the feedback. Considering that I recently offered to post a patch that would do essentially the same thing, I can't have any objections to this. ;-) But just to make sure I'm not missing anything in the patches, please let me know if any of the following is incorrect. What they do is remove the fs part of relayfs and move the remaining code into a single file, while leaving everthing else basically intact i.e. the relayfs kernel API remains the same and existing clients would only need to make relatively minor changes: - find a new home for their relay files i.e. sysfs, debufs or procfs. - replace any relayfs-specific code with their counterparts in the new filesystem i.e. directory creation/removal, non-relay ('control') file creation/removal. - change userspace apps to look for the relay files in the new filesystem instead of relayfs e.g. change /relay/* to /sys/* in the relay file pathnames. Although I personally don't have any problems with doing this, I've added some of the authors of current relayfs applications to the cc: list in case they might have any objections to it. The major relayfs applications I'm aware of are: - blktrace, currently in the -mm tree. This could probably move its relayfs files to sysfs using your new interface. - LTT, not sure where LTT would want to move. - systemtap. sytemtap uses relayfs as one possible transport. The other is a proc-based transport, so logically it would make sense for systemtap to move its relay files to /proc. Tom
From: Paul Mundt [email blocked] Subject: Re: [PATCH, RFC] sysfs: relay channel buffers as sysfs attributes Date: Mon, 20 Feb 2006 13:17:28 +0200 On Sun, Feb 19, 2006 at 11:29:23PM -0600, Tom Zanussi wrote: > But just to make sure I'm not missing anything in the patches, please > let me know if any of the following is incorrect. What they do is > remove the fs part of relayfs and move the remaining code into a > single file, while leaving everthing else basically intact i.e. the > relayfs kernel API remains the same and existing clients would only > need to make relatively minor changes: > > - find a new home for their relay files i.e. sysfs, debufs or procfs. > > - replace any relayfs-specific code with their counterparts in the new > filesystem i.e. directory creation/removal, non-relay ('control') > file creation/removal. > > - change userspace apps to look for the relay files in the new > filesystem instead of relayfs e.g. change /relay/* to /sys/* > in the relay file pathnames. > Yes, that's correct. With the patch I posted in reply to Dave it's also possible to keep the relayfs mount point that just wraps in to the new CONFIG_RELAY abstraction. Going this route (in place of the last 2 patches of my patch set) might make the migration a little less painful, but as there are no in-tree users for the kernel side, it's debatable whether keeping it around is worthwhile at all. The changes needed both in the kernel and user space client side are quite trivial anyways. Likewise it's also possible for the sysfs attribute patch to be applied first with the rest outstanding to let people start switching to the sysfs interface. Either way, as there seems to be enough interest in this, it would be nice to get it in to -mm for testing at least, people can follow blktrace's example on working with CONFIG_RELAY. The first 3 patches in my series can be used in addition to the follow-patch to Dave for introducing the CONFIG_RELAY interface while retaining legacy relayfs behaviour, so this might be the best way to go for -mm testing. The later patches can be rolled in once the few users have switched, if it's deemed that they're actually relevant as users. I'm fine with either approach, as long as I can use CONFIG_RELAY without having another mount point to deal with. > Although I personally don't have any problems with doing this, I've > added some of the authors of current relayfs applications to the cc: > list in case they might have any objections to it. The major relayfs > applications I'm aware of are: > > - blktrace, currently in the -mm tree. This could probably move its > relayfs files to sysfs using your new interface. > Jens pointed out that this should be quite easy, so that shouldn't be an issue. > - LTT, not sure where LTT would want to move. > LTT is already quite invasive on the kernel side. If they're interested in continually using relayfs rather than switching to something else, then they can use the patch I posted to Dave for stripping relayfs down to work with CONFIG_RELAY. This leaves relayfs implemented in 312 lines, which seems painless enough for the people that care about the API being consistent. > - systemtap. sytemtap uses relayfs as one possible transport. The > other is a proc-based transport, so logically it would make sense > for systemtap to move its relay files to /proc. Again, this is out-of-tree, and the kernel side changes to accomodate some other file system are trivial. If that's all the users, then fixing these up would seem to be the better alternative to leaving in a wrapper file system that people might want to use for future interfaces.
From: Paul Mundt [email blocked] Subject: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 23:07:33 +0200 This is a small patch set for getting rid of relayfs, and moving the core of its functionality to kernel/relay.c. The API is kept consistent for everything but the relayfs-specific bits, meaning people will have to use other file systems to implement relay channel buffers. There's support for a sysfs relay attribute (resent with a signed-off-by tag and cleaned up description), and it's also trivial to do via debugfs. Patches are ordered by invasiveness, the first few start out relatively harmless, with the later ones removing relayfs entirely. There's definitely room for cleanup and improvement in the new relay.c, but one thing at a time.. Tested with sysfs. Against current git. To get an idea of the total amount of changes: --- fs/relayfs/Makefile | 4 fs/relayfs/buffers.c | 190 --------- fs/relayfs/buffers.h | 12 fs/relayfs/inode.c | 581 ---------------------------- fs/relayfs/relay.c | 482 ----------------------- fs/relayfs/relay.h | 9 include/linux/relayfs_fs.h | 290 -------------- fs/Kconfig | 12 fs/Makefile | 1 fs/sysfs/Makefile | 3 fs/sysfs/dir.c | 28 + fs/sysfs/inode.c | 5 fs/sysfs/relay.c | 72 +++ include/linux/relay.h | 281 +++++++++++++ include/linux/relayfs_fs.h | 3 include/linux/sysfs.h | 33 + init/Kconfig | 11 kernel/Makefile | 1 kernel/relay.c | 923 ++++++++++++++++++++++++++++++++++++++++++++- 19 files changed, 1349 insertions(+), 1592 deletions(-)
From: Paul Mundt [email blocked] Subject: [PATCH 1/5] sysfs: relay channel buffers as sysfs attributes Date: Sun, 19 Feb 2006 23:07:57 +0200 Now with relayfs integrated and the relay_file_operations exported for use by other file systems, add a sysfs attribute for setting up channel buffers. The conventional relayfs doesn't make a lot of sense for the use cases where there are multiple devices to stream data from, particularly if they're already mapped out through the driver model. Rather than duplicating device enumeration, simply adding this as an attribute seems to work reasonably well. Tom did some work on the rchan_callbacks for more easily implementing relay files in other file systems, and it would be nice to use this in a non-debug context, without duplicating device enumeration in multiple locations.
From: Paul Mundt [email blocked] Subject: [PATCH 2/5] relay: Consolidate relayfs core into kernel/relay.c. Date: Sun, 19 Feb 2006 23:08:18 +0200 This abstracts the core parts needed for relay channel buffer management out of relayfs. With relay_file_operations exported, we can trivially support relay channels through other file systems (like sysfs and debugfs). While we're at it, get rid of some of the outstanding relayfs references from the code, as there's nothing file system dependent in the new relay abstraction.
From: Paul Mundt &[email blocked] Subject: [PATCH 3/5] sysfs: Update relay file support for generic relay API. Date: Sun, 19 Feb 2006 23:08:32 +0200 Now with the introduction of CONFIG_REPLAY, switch to using that instead of CONFIG_REPLAYFS_FS.
From: Paul Mundt [email blocked] Subject: [PATCH 4/5] relayfs: Remove relayfs in favour of CONFIG_RELAY. Date: Sun, 19 Feb 2006 23:08:57 +0200 Now that the relayfs core is abstracted as CONFIG_RELAY, and sysfs has a method for creating relay channel buffers as an attribute, relayfs can go away.
From: Paul Mundt [email blocked] Subject: [PATCH 5/5] relay: relay header cleanup. Date: Sun, 19 Feb 2006 23:09:14 +0200 Rename relayfs_fs.h to relay.h, drop some left over relayfs cruft, and update the various users of it.
From: Dave Jones [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 16:21:22 -0500 On Sun, Feb 19, 2006 at 11:07:33PM +0200, Paul Mundt wrote: > This is a small patch set for getting rid of relayfs, and moving the core of > its functionality to kernel/relay.c. The API is kept consistent for everything > but the relayfs-specific bits, meaning people will have to use other file > systems to implement relay channel buffers. What about the userspace visible API for things already using relayfs, like systemtap ? Whilst technically these patches may make sense, yanking the rug underneath applications as soon as they've started using it without warning or a migration period doesn't sound too good an idea. It's taken us *years* to try and get rid of devfs, why should relayfs get ripped out any quicker, when it has valid users? Dave
From: Paul Mundt [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 23:27:48 +0200 On Sun, Feb 19, 2006 at 04:21:22PM -0500, Dave Jones wrote: > On Sun, Feb 19, 2006 at 11:07:33PM +0200, Paul Mundt wrote: > > This is a small patch set for getting rid of relayfs, and moving the core of > > its functionality to kernel/relay.c. The API is kept consistent for everything > > but the relayfs-specific bits, meaning people will have to use other file > > systems to implement relay channel buffers. > > What about the userspace visible API for things already using relayfs, > like systemtap ? Whilst technically these patches may make sense, > yanking the rug underneath applications as soon as they've started > using it without warning or a migration period doesn't sound too good an idea. > Yes, that's why this is only done in the last two patches. The rest are completely independent and don't interfere with the API (well, ok, FIX_SIZE() needs to be dropped in fs/relayfs if it's fetching it from the header, but that's it). > It's taken us *years* to try and get rid of devfs, why should relayfs > get ripped out any quicker, when it has valid users? > I'm not advocating its removal, I'm more interested in using it from sysfs and debugfs where it makes more sense. Splitting up CONFIG_RELAY and CONFIG_RELAYFS_FS will allow for both, and it'll also be possible to migrate the existing CONFIG_RELAYFS_FS code to use CONFIG_RELAY without breaking the userspace APIs if we want to get rid of the duplication. The only issue will be wrapping up the default callbacks, but that's a trivial ifdef.
From: Paul Mundt [email blocked] Subject: [PATCH] relayfs: Convert to generic relay API. Date: Sun, 19 Feb 2006 23:54:32 +0200 On Sun, Feb 19, 2006 at 11:27:48PM +0200, Paul Mundt wrote: > On Sun, Feb 19, 2006 at 04:21:22PM -0500, Dave Jones wrote: > > On Sun, Feb 19, 2006 at 11:07:33PM +0200, Paul Mundt wrote: > > > This is a small patch set for getting rid of relayfs, and moving the core of > > > its functionality to kernel/relay.c. The API is kept consistent for everything > > > but the relayfs-specific bits, meaning people will have to use other file > > > systems to implement relay channel buffers. > > > > What about the userspace visible API for things already using relayfs, > > like systemtap ? Whilst technically these patches may make sense, > > yanking the rug underneath applications as soon as they've started > > using it without warning or a migration period doesn't sound too good an idea. > > > Yes, that's why this is only done in the last two patches. The rest are > completely independent and don't interfere with the API (well, ok, > FIX_SIZE() needs to be dropped in fs/relayfs if it's fetching it from the > header, but that's it). > > > It's taken us *years* to try and get rid of devfs, why should relayfs > > get ripped out any quicker, when it has valid users? > > > I'm not advocating its removal, I'm more interested in using it from > sysfs and debugfs where it makes more sense. Splitting up CONFIG_RELAY > and CONFIG_RELAYFS_FS will allow for both, and it'll also be possible to > migrate the existing CONFIG_RELAYFS_FS code to use CONFIG_RELAY without > breaking the userspace APIs if we want to get rid of the duplication. The > only issue will be wrapping up the default callbacks, but that's a > trivial ifdef. And here's a patch doing that.. any other objections? This strips down relayfs to a minimum, removing everything already provided by CONFIG_RELAY. This leaves a userspace API compatible implementation on top of the new framework.
From: Christoph Hellwig [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 22:08:40 +0000 On Sun, Feb 19, 2006 at 04:21:22PM -0500, Dave Jones wrote: > On Sun, Feb 19, 2006 at 11:07:33PM +0200, Paul Mundt wrote: > > This is a small patch set for getting rid of relayfs, and moving the core of > > its functionality to kernel/relay.c. The API is kept consistent for everything > > but the relayfs-specific bits, meaning people will have to use other file > > systems to implement relay channel buffers. > > What about the userspace visible API for things already using relayfs, There's no existing in-tree user of relayfs.
From: Dave Jones [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 17:13:30 -0500 On Sun, Feb 19, 2006 at 10:08:40PM +0000, Christoph Hellwig wrote: > > What about the userspace visible API for things already using relayfs, > There's no existing in-tree user of relayfs. wtf ? since when has userspace been 'in-tree' ? Dave
From: Christoph Hellwig [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 22:17:24 +0000 On Sun, Feb 19, 2006 at 05:13:30PM -0500, Dave Jones wrote: > On Sun, Feb 19, 2006 at 10:08:40PM +0000, Christoph Hellwig wrote: > > > > What about the userspace visible API for things already using relayfs, > > There's no existing in-tree user of relayfs. > > wtf ? since when has userspace been 'in-tree' ? relayfs is a kernel component. to create any user-visible thing in relayfs some kernel components needs to use it. no single kernel component does currently, and the we've only put it in after Andrew agreed that this won't stop us fixing things in just because it's in yet in the long discussion we had before merging it.
From: Dave Jones [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 17:29:43 -0500 On Sun, Feb 19, 2006 at 10:17:24PM +0000, Christoph Hellwig wrote: > On Sun, Feb 19, 2006 at 05:13:30PM -0500, Dave Jones wrote: > > On Sun, Feb 19, 2006 at 10:08:40PM +0000, Christoph Hellwig wrote: > > > > > > What about the userspace visible API for things already using relayfs, > > > There's no existing in-tree user of relayfs. > > > > wtf ? since when has userspace been 'in-tree' ? > > relayfs is a kernel component. to create any user-visible thing in relayfs > some kernel components needs to use it. no single kernel component does > currently systemtap uses kprobes to export data through relayfs. Dave
From: Christoph Hellwig [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 22:48:20 +0000 On Sun, Feb 19, 2006 at 05:29:43PM -0500, Dave Jones wrote: > systemtap uses kprobes to export data through relayfs. systemtap has never been anywhere near mainline despit me telling the ibm folks to get the non-braindead parts in, so it simply doesn't matter.
From: Dave Jones [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Sun, 19 Feb 2006 17:54:37 -0500 On Sun, Feb 19, 2006 at 10:48:20PM +0000, Christoph Hellwig wrote: > On Sun, Feb 19, 2006 at 05:29:43PM -0500, Dave Jones wrote: > > systemtap uses kprobes to export data through relayfs. > > systemtap has never been anywhere near mainline despit me telling the > ibm folks to get the non-braindead parts in, so it simply doesn't matter. the kernel bits that matter are generated at runtime. I don't see anything particularly interesting in the source tree that is worthwhile submitting. Dave
From: Christoph Hellwig [email blocked] Subject: Re: [PATCH 0/5] relay: Migrate from relayfs to a generic relay API. Date: Mon, 20 Feb 2006 16:03:13 +0000 On Sun, Feb 19, 2006 at 05:54:37PM -0500, Dave Jones wrote: > On Sun, Feb 19, 2006 at 10:48:20PM +0000, Christoph Hellwig wrote: > > On Sun, Feb 19, 2006 at 05:29:43PM -0500, Dave Jones wrote: > > > systemtap uses kprobes to export data through relayfs. > > > > systemtap has never been anywhere near mainline despit me telling the > > ibm folks to get the non-braindead parts in, so it simply doesn't matter. > > the kernel bits that matter are generated at runtime. > I don't see anything particularly interesting in the source tree > that is worthwhile submitting. Various helpers to write kprobes in C, instead of the funky limited functionality scripting language of the day. But anyway, if systemtap folks really need a relayfs filesystem type they're totally free under the gpl to grab the code Paul posted to recreate relayfs ontop of kernel/relay.c and include it in their codebase, no need to bloat the kernel tree with unused code.
Email filter broken
It looks like the filter for emails is a bit broken, in the beginning it says:
From: Paul Mundt [email blocked]
But some emails later his address shows up.
Probably the '-' in the domai
Probably the '-' in the domainname...
My site :D
Nice Site !