Home » Mailing list archives » linux-kernel » 2008 » July » 7

[1/3] POHMELFS: VFS trivial change.

Previous thread: none

Next thread: [PATCH] drivers/mtd/chips/jedec_probe.c (linux-2.6.25.10) by Jerry Hicks on Monday, July 7, 2008 - 2:16 pm. (1 message)
[view original message]
From: Evgeniy Polyakov <johnpol@...>
To: <linux-kernel@...>
Cc: <netdev@...>, <linux-fsdevel@...>
Subject: [0/3] POHMELFS high performance network filesystem. First steps in parallel processing.
Date: Monday, July 7, 2008 - 2:07 pm

Hi.


I'm pleased to announce POHMEL high performance network parallel distributed filesystem.

POHMELFS stands for Parallel Optimized Host Message Exchange Layered File System.


Development status can be tracked in filesystem section [1].


This is a high performance network filesystem with local coherent cache of data

and metadata. Its main goal is distributed parallel processing of data.

System supports strong transaction model with failover recovery, allows to

encrypt/hash whole data channel and perform read load balancing and

write to multiple servers in parallel.


This release brings following features:

 * Strong crypto support. One can specify encryption method (like

 	cbc(aes), hash or digest, or all of them to be performed on

	whole data channel (except headers). If special mount option is

	specified, then mount will file if server does not support

	requested by client crypto options. By defalt that crypto

	options will be skipped.

 * Bug fixes.


Very likely it is one of the last non-bug-fixing release of the kernel

client side, next release will incorporate features, needed for distributed

parallel data processing (like ability to add new servers via network

command from another servers), so most of the work will be devoted to server

code.


Basic POHMELFS features:

 * Local coherent (notes [2]) cache for data and metadata).

 * Completely async processing of all events (hard and symlinks are the only

    	exceptions) including object creation and data reading/writing.

 * Flexible object architecture optimized for network processing. Ability to

    	create long pathes to object and remove arbitrary huge directoris in

	single network command.

 * High performance is one of the main design goals.

 * Very fast and scalable multithreaded userspace server. Being in userspace

    	it works with any underlying filesystem and still is much faster than

	async ni-kernel NFS one.

 * Client is able to switch between different servers (if one goes down,

	client automatically re...
[ message continues ]
[view original message]
From: Evgeniy Polyakov <johnpol@...>
To: <linux-kernel@...>
Cc: <netdev@...>, <linux-fsdevel@...>
Subject: [0/3] POHMELFS high performance network filesystem. Strong crypto support.
Date: Monday, July 7, 2008 - 2:15 pm

Subject should be "POHMELFS high performance network filesystem. Strong

crypto support."


This release brings ability to encrypt data channel in case of

distributed filesystem created on top of uncontrolled channels.


One can setup different crypto algorithms, which will be autoconfigured

at mount time (depending on options it will or will not fail, if server

does not support requested algorithms, if administrator requested not to

fail, system will disable appropriate algorithm usage).


IOzone performance testing showed performance drop in case of

cbc(aes)+sha1 crypto operations, but system still behaves faster than

plain in-kernel async NFS. Nice picture in attach (or via link at:

http://tservice.net.ru/~s0mbre/blog/devel/fs/2008_07_07.html)


Plain read/write performance link:

http://tservice.net.ru/~s0mbre/blog/devel/fs/2008_06_25.html


--

	Evgeniy Polyakov
[view original message]
From: Evgeniy Polyakov <johnpol@...>
To: <linux-kernel@...>
Cc: <netdev@...>, <linux-fsdevel@...>
Subject: [2/3] POHMELFS: Documentation.
Date: Monday, July 7, 2008 - 2:10 pm

Design notes, usage cases and protocol description.


Signed-off-by: Evgeniy Polyakov <johnpol@2ka.mipt.ru>


diff --git a/Documentation/filesystems/pohmelfs/design_notes.txt b/Documentation/filesystems/pohmelfs/design_notes.txt

new file mode 100644

index 0000000..0b20a47

--- /dev/null

+++ b/Documentation/filesystems/pohmelfs/design_notes.txt

@@ -0,0 +1,70 @@

+POHMELFS: Parallel Optimized Host Message Exchange Layered File System.

+

+		Evgeniy Polyakov <johnpol@2ka.mipt.ru>

+

+Homepage: http://tservice.net.ru/~s0mbre/old/?section=projects&item=pohmelfs

+

+It was first started as network filesystem with coherent local data and metadata caches,

+but it is being evolved into parallel distibuted filesystem now.

+

+Main features of this FS include:

+ * Local coherent (notes cache for data and metadata:

+ 	http://tservice.net.ru/~s0mbre/blog/devel/fs/2008_05_17.html)

+ * Completely async processing of all events (hard, symlinks and rename are the

+ 	only exceptions) including object creation and data reading and writing.

+ * Flexible object architecture optimized for network processing.

+ 	Ability to create long pathes to object and remove arbitrary huge

+	directoris in single network command.

+	(like removing the whole kernel tree via single network command).

+ * Very high performance.

+ * Fast and scalable multithreaded userspace server. Being in userspace it works

+ 	with any underlying filesystem and still is much faster than async in-kernel NFS one.

+ * Client is able to switch between different servers (if one goes down, client

+ 	automatically reconnects to second and so on).

+ * Transactions support. Full failover for all operations.

+ 	Resending transactions to different servers on timeout or error.

+ * Read requests (data read, directory listing, lookup requests) balancing between multiple servers.

+ * Write requests are sent to multiple servers and completed only when all of them sent an ack.

+ * Ability to add and/or remove servers from working set at run-t...
[ message continues ]
[view original message]
From: Pavel Machek <pavel@...>
To: Evgeniy Polyakov <johnpol@...>
Cc: <linux-kernel@...>, <netdev@...>, <linux-fsdevel@...>
Subject: Re: [2/3] POHMELFS: Documentation.
Date: Saturday, July 12, 2008 - 3:01 am

Hmm... this is little funny.


Which endianity?


							Pavel

--

(english) http://www.livejournal.com/~pavelmachek

(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

--
[view original message]
From: Evgeniy Polyakov <johnpol@...>
To: Pavel Machek <pavel@...>
Cc: <linux-kernel@...>, <netdev@...>, <linux-fsdevel@...>
Subject: Re: [2/3] POHMELFS: Documentation.
Date: Saturday, July 12, 2008 - 3:26 am

Hi.


There are people who belive that most with exceptions is still all :)


Big endian in the network. CPU endian on both ends.


--

	Evgeniy Polyakov

--
[view original message]
From: Evgeniy Polyakov <johnpol@...>
To: <linux-kernel@...>
Cc: <netdev@...>, <linux-fsdevel@...>
Subject: [1/3] POHMELFS: VFS trivial change.
Date: Monday, July 7, 2008 - 2:08 pm

Signed-off-by: Evgeniy Polyakov <johnpol@2ka.mipt.ru>


diff --git a/mm/filemap.c b/mm/filemap.c

index 07e9d92..57beb4b 100644

--- a/mm/filemap.c

+++ b/mm/filemap.c

@@ -495,6 +495,7 @@ int add_to_page_cache_lru(struct page *page, struct address_space *mapping,

 		lru_cache_add(page);

 	return ret;

 }

+EXPORT_SYMBOL_GPL(add_to_page_cache_lru);


 #ifdef CONFIG_NUMA

 struct page *__page_cache_alloc(gfp_t gfp)

@@ -610,6 +611,7 @@ int __lock_page_killable(struct page *page)

 	return __wait_on_bit_lock(page_waitqueue(page), &wait,

 					sync_page_killable, TASK_KILLABLE);

 }

+EXPORT_SYMBOL_GPL(__lock_page_killable);


 /**

  * __lock_page_nosync - get a lock on the page, without calling sync_page()


--

	Evgeniy Polyakov

--
[view original message]
From: Evgeniy Polyakov <johnpol@...>
To: <linux-kernel@...>
Cc: <netdev@...>, <linux-fsdevel@...>
Subject: [3/3] POHMELFS: core files.
Date: Monday, July 7, 2008 - 2:11 pm

POHMELFS core.


Signed-off-by: Evgeniy Polyakov <johnpol@2ka.mipt.ru>


diff --git a/Documentation/filesystems/pohmelfs/design_notes.txt b/Documentation/filesystems/pohmelfs/design_notes.txt

new file mode 100644

index 0000000..0b20a47

--- /dev/null

+++ b/Documentation/filesystems/pohmelfs/design_notes.txt

@@ -0,0 +1,70 @@

+POHMELFS: Parallel Optimized Host Message Exchange Layered File System.

+

+		Evgeniy Polyakov <johnpol@2ka.mipt.ru>

+

+Homepage: http://tservice.net.ru/~s0mbre/old/?section=projects&item=pohmelfs

+

+It was first started as network filesystem with coherent local data and metadata caches,

+but it is being evolved into parallel distibuted filesystem now.

+

+Main features of this FS include:

+ * Local coherent (notes cache for data and metadata:

+ 	http://tservice.net.ru/~s0mbre/blog/devel/fs/2008_05_17.html)

+ * Completely async processing of all events (hard, symlinks and rename are the

+ 	only exceptions) including object creation and data reading and writing.

+ * Flexible object architecture optimized for network processing.

+ 	Ability to create long pathes to object and remove arbitrary huge

+	directoris in single network command.

+	(like removing the whole kernel tree via single network command).

+ * Very high performance.

+ * Fast and scalable multithreaded userspace server. Being in userspace it works

+ 	with any underlying filesystem and still is much faster than async in-kernel NFS one.

+ * Client is able to switch between different servers (if one goes down, client

+ 	automatically reconnects to second and so on).

+ * Transactions support. Full failover for all operations.

+ 	Resending transactions to different servers on timeout or error.

+ * Read requests (data read, directory listing, lookup requests) balancing between multiple servers.

+ * Write requests are sent to multiple servers and completed only when all of them sent an ack.

+ * Ability to add and/or remove servers from working set at run-time from userspace (via netlink,

+ 	...
[ message continues ]
Previous thread: none

Next thread: [PATCH] drivers/mtd/chips/jedec_probe.c (linux-2.6.25.10) by Jerry Hicks on Monday, July 7, 2008 - 2:16 pm. (1 message)