login
Search
Hello All,
Our team want to develop a distributed file system, a
file system like NFS or in other words a remote file
access system.
In our design we have multiple clients and a single
server. I want to identify main issues involved. What
I found after doing some study is this:
(1) Handling of multiple clients (concurrent
read/write problem)
(2) Continuation of (1) i.e. the distributed lock
management
(3) For performance reasons caching on the client side
and the cache coherency problem
(this can be avoided for our prototype release)
(4) Data integrity problem.
To solve these issues, I think I have to study NFS,
AFS and coda. Also I am doing general googling for
finding out distributed file system issues.
Since both linux and fs (especially a distributed one)
are new to me, I want help in this regard that how I
start this development of a prototype and which
documents/books to study for this purpose.
Thanks,
Uzair Lakhani,
Karachi, Pakistan.
Why?
Why do you want to make a new distributed filesystem, except for the fun of it?
From my point of view, the ma
From my point of view, the main problem of every distributed file system is file system structure integrity -- be it on a sub-file (data) level or just at the file/directory structure level.
To allow for scaling, one has to break up the consistency. For example, you can divide the distributed file system into domains and each of those domains would have its own master server that ensures consistency. The scalability depends at last on the scalability of the specific problem at hand.
If you want to create somethi
If you want to create something really new and not just a clone of what already exists half a dozen times, then, maybe, have a look at my page at:
https://stier.cable.nu/~moinmoin/MarksWiki/Programming/AsynchronouslyDis...
Why a new file system
Hello All,
Thanks for all the reply I received. As you may have noticed that I had earlier asked some questions regarding cluster file system. Basically we have to make a simple fail-over and performance cluster file system.
In order to acheive that target and to find out the different issues involved, we have decided to first make a distributed file system ourselves to find out the issues involved. Then later extend it to a cluster file system.
So this is the basic reason to develop a new distributed file system.
Thanks,
Uzair Lakhani,
Karachi, Pakistan.
Why not use GFS (http://www.r
Why not use GFS (http://www.redhat.com/software/rha/gfs) as your clustering filesystem and export it using NFS?
There is no reason to re-invent the wheel, there are plenty of round objects already!
They need a GPL coder for
They need a GPL coder for exactly what you would like to do. Start with them take a look at
http://ceph.sourceforge.net/
Glusterfs
GlusterFS also looks like a nice and novel solution to this requirement?
GFS/OCFS is about sharing the block device. NFS/PNFS/GlusterFS is about sharing the filesystem (and DRBD seems like a nice solution to replicate block devices)
Also see elasticdrive. They make amazon s3 into a block device. The interesting bit is that they point out that you can use a Raid1 setup with an option to write mainly to the local block device and then lag writing to the S3 block device. This seems like a very interesting avenue... (one fast local block device replicated to a slower one elsewhere)
Ed W