The debate continues regarding the removal of the kernel space devfs in favor of the user space udev, though less people seem concerned about the idea. Linux devfs was written by Richard Gooch and merged into the 2.3.46 kernel in February of 2000. Greg KH began developing udev in 2001 [story], working to implement the same functionality as devfs, but in userspace. A resubmission of a patch intended to remove the unmaintained devfs from the kernel [story] spawned a renewed discussion of the potential merits of keeping devfs.

Mike Bell titled his arguments the "devfs vs udev FAQ from the other side", summarizing some differences between the two systems, and listing potential advantages of each. Robert Love [interview] contested some of his claims for devfs, pointing to the more modern features in udev, "what modern system, though, could survive without hotplug and sysfs and netlink?" Mike replied, "devfs's removal from the kernel isn't going to end my life, no. In fact it won't even stop me using linux. But it makes my job harder and more expensive, and as my little FAQ explains I don't feel I'm getting anything out of the deal."

From: Mike Bell [email blocked]
To:  linux-kernel
Subject: devfs vs udev FAQ from the other side
Date:	Wed, 14 Sep 2005 17:51:06 -0700

devfs vs udev
From the other side

Presuppositions (True of both udev and devfs):
1) Dynamic /dev is the way of the future, and a Good Thing
2) A single major/minor combination should have only a single device
node, its other names should be symlinks. If you don't do this, you
break locking on certain classes of applications, among other things.

The above are uncontentious as far as I know. I believe Greg KH has
stated both. If you feel otherwise, please explain why.

Differences:
1) devfs creates device nodes from kernel space, and creates symlinks
  for alternative names using a userspace helper. udev handles both tasks
  from user space, by exporting the information through a different
  kernel-generated filesystem.

devfs advantages over udev:
1) devfs is smaller
  Hey, I ran the benchmarks, I have numbers, something Greg never gave.
  Took an actual devfs system of mine and disabled devfs from the
  kernel, then enabled hotplug and sysfs for udev to run.  make clean
  and surprise surprise, kernel is much bigger. Enable netlink stuff and
  it's bigger still. udev is only smaller if like Greg you don't count
  its kernel components against it, even if they wouldn't otherwise need
  to be enabled. Difference is to the tune of 604164 on udev and 588466
  on devfs. Maybe not a lot in some people's books, but a huge
  difference from the claims of other people that devfs is actually
  bigger.

  And that's just the kernel. Then because your root is read-only you
  need an early userspace, and in regular userspace the udev binary, and
  its data files, all more wasted flash (you can shave it down by
  removing stuff you don't need, but that's just more work for the busy
  coder, and udev STILL loses on size).

  On the system in question (a real-world embedded system) the devfs
  solution requires no userspace helper except for two symlinks which
  were simply created manually in init, and could have been done away
  with if necessary.

2) devfs is faster
  Despite all the many tricks that can be used to speed up udev (static
  linking, netlink, etc) devfs is still dramatically faster. On a big,
  bloated, slow-booting distribution system you may not notice so much,
  but when even your slowest booting systems are interactive in under 5
  seconds using devfs, this is quite significant time loss.

3) devfs uses less memory
  Check free. sysfs alone does udev in and that's just the kernel stuff
  that's always there.
  
  Also, the user space stuff may not have to run at all times in all
  configurations, but on a system without swap and with long-running
  apps, all that matters is its PEAK memory usage. If my app takes x MB
  and my kernel takes y MB it doesn't MATTER that udev is only running
  for one second, I still need more than x+yMB of memory.


udev advantages over devfs:
1) udev has all sorts of spiffy features
  Sure, but having device nodes exported directly from the kernel in no
  way stops you from having those spiffy features. The problem is that
  udev is doing two separate tasks, and it's easy to confuse the one it
  should be doing with the one it shouldn't.

2) udev doesn't have policy in kernel space
  Well, that's a bit of a lie. sysfs has even stricter policy in kernel
  space. What he MEANS is that udev exchanges hard-coded but symlinkable
  /dev paths for hardcoded sysfs paths, moving the hard-coded kernel
  policy from one filesystem to another.

  This argument is really the only architectural reason to go with udev.
  At all. If you really believe that the ability to name your hard drive
  /dev/foobarbaz is vital, and absolutely can't live with merely having
  /dev/foobarbaz be a symlink to the real device node, then you need udev.
  The devfs way of handling this situation was a stupid, racey misfeature
  and rightly deserves to die horribly.

  That said, read my comments on why flexible /dev naming is actually a
  bad thing and think very, very carefully about whether you actually want
  this "feature" at all. Symlinks are your friend.

3) devfs is ugly
  Part of this is true, and part of this is just the perspective of
  certain people (Greg has this fascinating world view where code
  required for devfs is garbage, and code required for udev is core
  kernel code and doesn't count against udev, which allows him to say
  udev is smaller.)
  
  The legitimate comments about devfs being ugly... well, how many
  subsystems which have been largely untouched for similar periods of
  time aren't even uglier? TTY stuff? And it's very hard to find a
  maintainer for a subsystem when it's "obsolete", patches that change
  its behaviour aren't accepted, and certain people are so vocally
  opposed to its very existence. Who wants to throw away their time
  writing code that won't even be considered, only to be hated for it?

4) devfs is unsupported, udev isn't
  True that. And even people who've tried to maintain devfs get turned
  away. So unless this document causes a few people to reexamine the
  need to remove devfs, you can reasonably assume that udev will be the
  only way to run a linux system very shortly (static /dev is already on
  its last legs). Me, I'll be disappointed if this happens, because as
  the above document indicates, I still think kernel-exported /dev is
  better (and not because I'm a lazy user-space-hater, Greg. :) ).



From: Robert Love [email blocked]
Subject: Re: devfs vs udev FAQ from the other side
Date:	Wed, 14 Sep 2005 21:08:38 -0400

On Wed, 2005-09-14 at 17:51 -0700, Mike Bell wrote:

> devfs advantages over udev:
> 1) devfs is smaller
>   Hey, I ran the benchmarks, I have numbers, something Greg never gave.

Actually, there are not many numbers in this email.

>   Took an actual devfs system of mine and disabled devfs from the
>   kernel, then enabled hotplug and sysfs for udev to run.  make clean
>   and surprise surprise, kernel is much bigger. Enable netlink stuff and
>   it's bigger still. udev is only smaller if like Greg you don't count
>   its kernel components against it, even if they wouldn't otherwise need
>   to be enabled. Difference is to the tune of 604164 on udev and 588466
>   on devfs. Maybe not a lot in some people's books, but a huge
>   difference from the claims of other people that devfs is actually
>   bigger.

What modern system, though, could survive without hotplug and sysfs and
netlink?  You need to have those components, you want those features,
anyhow.

So your comparison is unrealistic.

Your user-space argument is better.  Is ndevfs not sufficient?

	Robert Love



From: Mike Bell [email blocked]
Subject: Re: devfs vs udev FAQ from the other side
Date:	Wed, 14 Sep 2005 19:09:36 -0700

On Wed, Sep 14, 2005 at 09:08:38PM -0400, Robert Love wrote:
> Actually, there are not many numbers in this email.

True, but two is more than zero. If they're going to do some good, I can
always make more to show exactly how much udev slows down a boot and
such. But at this point I have my doubts as to whether any benchmarks
will change people's minds.

> What modern system, though, could survive without hotplug and sysfs and
> netlink?  You need to have those components, you want those features,
> anyhow.

This is a modern system. As I took great pains to point out, that's an
actual embedded system whose .config I copied and slightly modified for
the test.

> So your comparison is unrealistic.

Not really. sysfs has a few uses now, but there are still virtually
no embedded applications of it. Hotplug is nice for things whose
hardware change, but useless for anything else (unless you need udev).
Even on my notebook I only use it for the event from my CF port.

Note that I'm not suggesting any of those features be removed from the
kernel, merely that they should count against udev's totals when the
system in question has no other use for them. And I don't see who else
is making the argument for a couple of kB apart from exactly the sort of
people who are disabling these features. There's a reason the -tiny
patchset introduced the ability to disable sysfs, it isn't always needed
and it does take up a lot of memory and bloat the kernel.

> Your user-space argument is better.  Is ndevfs not sufficient?

Nope, unfortunately. If you read my initial response I was quite
thrilled. But once I realized the limitations of ndevfs I changed my
tone. The devfs hooks are still required to provide appropriate names,
and it is these I'm most interested in saving.



From: Robert Love [email blocked]
Subject: Re: devfs vs udev FAQ from the other side
Date:	Wed, 14 Sep 2005 23:17:50 -0400

On Wed, 2005-09-14 at 19:09 -0700, Mike Bell wrote:

> Nope, unfortunately. If you read my initial response I was quite
> thrilled. But once I realized the limitations of ndevfs I changed my
> tone. The devfs hooks are still required to provide appropriate names,
> and it is these I'm most interested in saving.

If you can survive without modernism, why can't you survive without
devfs?  Why do you need it?

	Robert Love



From: Mike Bell [email blocked]
Subject: Re: devfs vs udev FAQ from the other side
Date:	Wed, 14 Sep 2005 21:13:34 -0700

On Wed, Sep 14, 2005 at 11:17:50PM -0400, Robert Love wrote:
> If you can survive without modernism,

Features I don't need are "modernism"? Who defines what's "modern" and
what's merely "useless"? Some things in newer kernels are very, very
useful. Even sysfs is useful on some of my systems. But no, I generally
try to avoid turning on features that I'm not even going to use on a
given system. Do you install things you're not going to use on your
machines? Maybe two or three different SQL servers, just to be
extra-modern? Especially when I'm paying for every megabyte of flash x1k
or x10k, I don't want to waste it on a bunch of unused kernel->userspace
interfaces just to prove how incredibly hip to the latest linux fads I
am.

> why can't you survive without devfs?  Why do you need it?

Why can't you read any of the many threads on this subject where I
explained this already? :)

devfs's removal from the kernel isn't going to end my life, no. In fact
it won't even stop me using linux. But it makes my job harder and more
expensive, and as my little FAQ explains I don't feel I'm getting
anything out of the deal (remember, udev's naming features are different
to udev's device node creation, there's no reason you can't have one
without the other).

• Archive of above thread
• KernelTrap interview with Robert Love