Hurd: UnionFS Effort Replaces ShadowFS Effort

Submitted by Anonymous
on December 8, 2002 - 10:11am

Moritz Schulte announced that he has stopped working on the ShadowFS file system for the GNU/Hurd. Instead, he has begun working on a new, better designed filesystem under the name UnionFS, a name change suggested by Richard Stallman. Moritz says, "Compared with my former implementation, this one uses a completely new design approach."

One major improvement UnionFS offers over his previous ShadowFS implementation is the addition of caching. A second improvment is that you can now add and remove directory trees from/to the united tree during runtime with the fsysopts program, even the tree that underlies the node where UnionFS itself sits. The current code is still very much under development and certainly has its bugs and limitations, but seems to work well enough at least for some purposes when used in read-only mode.


From: Moritz Schulte
To: bug-hurd mailing list
Subject: Unionfs
Date: Sat, 07 Dec 2002 19:06:09 +0100

Hello,

  after not doing any work on my former shadowfs implementation
anymore for many months, I started working on a new implementation
(`unionfs') few weeks ago.  It is quite uncomplete at the moment and
has not few problems.  Compared with my former implementation, this
one uses a completely new design approach.

  I have just checked in my latest sources into the Hurd Extras CVS
repository; it can be checked out with the command:

 $ cvs -d:anoncvs@subversions.gnu.org:/cvsroot/hurdextras co unionfs

Have fun/Happy hacking.

Thanks,
                moritz

_

From: Wolfgang Jaehrling
To: bug-hurd mailing list
Subject: Re: Unionfs
Date: Sun, 8 Dec 2002 08:35:21 +0100

On Sat, Dec 07, 2002 at 07:06:09PM +0100, Moritz Schulte wrote:
> Have fun/Happy hacking.

I was just playing around with it a bit and glancing over the code;
when compiling, I got the messages:

gcc -o unionfs main.o node.o lnode.o ulfs.o options.o 
  ncache.o netfs.o lib.o -lnetfs -lfshelp -liohelp -lthreads 
  -lports -lihash -lshouldbeinlibc
/usr/bin/ld: netfs.o(.debug_info+0x6399):
  unresolvable relocation against symbol `_netfs_translator_callback1'
/usr/bin/ld: netfs.o(.debug_info+0x63ac):
  unresolvable relocation against symbol `_netfs_translator_callback2'

However, it did work nevertheless (i.e. the unionfs binary was created
and could be started).

I noticed that lnode_ref_remove() and lnode_uninstall() recursively
call each other, where lnode_uninstall() calls lnode_ref_remove() on
node->dir, which should be locked; I could not find anything that
indicates that it will be.

Also, I find it a bit unfortunate that a simple `ls' triggers this
already:

wj@hurd:~/unionfs$ settrans -ac foo unionfs .. /
wj@hurd:~/unionfs$ ls foo/unionfs/
ls: foo/unionfs/foo: Too many levels of symbolic links

And of course, I just had to try compiling Unionfs inside of a
Unionfs. :-) The result was:

wj@hurd:~/unionfs/foo/unionfs$ make
make: getcwd: : No such file or directory
gcc -Wall -g -D_FILE_OFFSET_BITS=64 -std=gnu99 -DDEBUG -c main.c
Assembler messages:
FATAL: can't create main.o: Invalid argument
make: *** [main.o] Error 1

More feedback coming soon.

Cheers,
GNU/Wolfgang

_

From: Moritz Schulte
To: bug-hurd mailing list
Subject: Re: Unionfs
Date: Sun, 08 Dec 2002 09:04:11 +0100

Wolfgang Jaehrling writes:

> /usr/bin/ld: netfs.o(.debug_info+0x6399): unresolvable relocation
> against symbol `_netfs_translator_callback1'

I am not sure what this is; I don't see it here.

> I noticed that lnode_ref_remove() and lnode_uninstall() recursively
> call each other, where lnode_uninstall() calls lnode_ref_remove() on
> node->dir, which should be locked; I could not find anything that
> indicates that it will be.

Hmm, I will look into that.

> Also, I find it a bit unfortunate that a simple `ls' triggers this
> already:
>
> wj@hurd:~/unionfs$ settrans -ac foo unionfs .. /
> wj@hurd:~/unionfs$ ls foo/unionfs/
> ls: foo/unionfs/foo: Too many levels of symbolic links

Of course; that is exactly the problem we were speaking about on IRC.
If you would be allowed to lookup foo/unionfs/foo, this would lead to
endless recursion - wouldn't be so nice for e.g. the locatedb-find
process.

Thanks for your comments.

                moritz

_

From: Wolfgang Jaehrling
To: bug-hurd mailing list
Subject: Re: Unionfs
Date: Sun, 8 Dec 2002 09:17:33 +0100

On Sun, Dec 08, 2002 at 09:04:11AM +0100, Moritz Schulte wrote:
> > Also, I find it a bit unfortunate that a simple `ls' triggers this
> > already:
> >
> > wj@hurd:~/unionfs$ settrans -ac foo unionfs .. /
> > wj@hurd:~/unionfs$ ls foo/unionfs/
> > ls: foo/unionfs/foo: Too many levels of symbolic links
>
> Of course; that is exactly the problem we were speaking about on IRC.
> If you would be allowed to lookup foo/unionfs/foo, this would lead to
> endless recursion - wouldn't be so nice for e.g. the locatedb-find
> process.

Still, I find it a bit unfortunate that a simple `ls' triggers this
already. ;-)

How about giving it the appearance of an empty directory instead?
Then dired mode and `ls -l' etc. would not fail when looking at the
parent directory of the node in question; I don't think it would cause
any harm.

Cheers,
GNU/Wolfgang

Typo

Anonymous
on
December 8, 2002 - 6:16pm

May I suggest fixing the typo in Moritz' last name before anyone sees it? 8-)

re: Typo

Jeremy
on
December 8, 2002 - 6:33pm

Thanks. Fixed. :)

Why a new file system?

Anonymous
on
December 8, 2002 - 8:46pm

Why not porting an existing file system such as ReiserFS or ext3 to Grub? It seems that creating a brand new FS is re-inventing the wheel at this point since many advanced FS are pretty mature now.
Just my 2c point of view...

It's not really a filesystem as such...

coriordan
on
December 9, 2002 - 12:36am

I run a Hurd based GNU system and try to keep informed on the
development process. From what I know, ShadowFS wasn't really
a filesystem in that it didn't store files. What it did was
map new directory names to existing filesystem directorys.

I've never used ShadowFS but I believe one of it's purposes was
to emulate the directory layout of other systems. One developer
didn't like that Hurd based systems have /usr as a symlink to /
so he wanted to use ShadowFS to make his system appear to be layed
out the way he liked it.

A bit like a virtual FS, a layer over other (real) FS's.

Ciaran O'Riordan

Why not eat cheese?

Anonymous
on
December 9, 2002 - 1:37pm

Hi!

Porting a stored file system like ext3fs or ReiserFS to GRUB (or did you mean GNU/Hurd?) has nothing to do with UnionFS. UnionFS is not a disk-based file system, it is more similar to a virtual file system (like /proc on GNU/Linux). It unites several directory trees, i.e. just gives you a new view on data that also exists elsewhere, a feature that is useful for a lot of things, including a Life-CD system that can transparently record changes on a floppy via Copy-On-Write, for example. This has nothing to do with reinventing the wheel. We are interested in a port of ReiserFS and especially ext3fs, of course.

Cheers,
GNU/Wolfgang

cool

Hiryu
on
December 8, 2002 - 9:04pm

I'd like to see more stories about the hurd... Though I understand if there simply just aren't that many right now.

maybe I'll write one (or you?)

coriordan
on
December 9, 2002 - 12:48am

Hmm. I installed Debian GNU/Hurd (J2) recently and was very
surprised at how easy it was. So surprised that I was thinking
of writing a story about my experience.

I still have my few problems, I don't have ppp or X working. ppp
doesn't really work at all under the Hurd, this *may* be solved
by the move to GNU-Mach2 (OSKit Mach). I haven't really made an
effort to get X working, I only use it for browsing the internet.

Hopefully in a week or two I'll write & submit that story. In the
mean time I'd suggest give the Hurd a try. When I'm using Emacs
or Vim (or GCC, guile, makeinfo etc.) it's easy to forget whether
I booted GNU or GNU/Linux.

Ciaran O'Riordan

amazing =)

Anonymous
on
December 9, 2002 - 2:54pm

Great advance.
Proud of you.

GNU/Bunny

related reading, "Union Mounts in 4.4BSD-Lite" USENIX paper

salo
on
December 9, 2002 - 7:41pm
     Author: Jan-Simon Pendry                                                                                    
     Author: Marshall Kirk McKusick                                                                              
      Title: Union Mounts in 4.4BSD-Lite
      Pages: 25-33                                                                                              
  Publisher: USENIX                                                                                             
Proceedings: USENIX Conference Proceedings                                                                      
       Date: January 16-20, 1995                                                                           
   Location: New Orleans, LA                                                                                    
Institution: Sequent Corporation                                                                                
Institution: Author and Consultant                                                                              

or --bind mounts with linux

Anonymous
on
December 11, 2002 - 12:38am

with the new 2.5 vfs layer bind mounts are faster and more flexible than ever

How flexible?

Anonymous
on
December 11, 2002 - 4:01pm

How flexible are they exactly?

How flexible?

Anonymous
on
December 12, 2002 - 11:52am

More than ever.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.