Re: New distributed storage release.

Previous thread: Controlling backlight on thinkpad x60 by Pavel Machek on Monday, September 8, 2008 - 8:10 am. (7 messages)

Next thread: [PATCH] x86: io-apic - do not use KERN_DEBUG marker too much by Cyrill Gorcunov on Monday, September 8, 2008 - 8:38 am. (6 messages)
From: Evgeniy Polyakov
Date: Monday, September 8, 2008 - 8:22 am

Hi.

I am pleased to announce new Distributed Storage (DST) project release.

DST is a block layer network device, which among others has following
features:

 * Kernel-side client and server. No need for any special tools for
	data processing (like special userspace applications) except
	for configuration.
 * Bullet-proof memory allocations via memory pools for all temporary
 	objects (transaction and so on).
 * Zero-copy sending (except header) if supported by device
 	using sendpage().
 * Failover recovery in case of broken link
 	(reconnection if remote node is down).
 * Full transaction support (resending of the failed transactions
 	on timeout of after reconnect to failed node).
 * Dynamically resizeable pool of threads used for data receiving
 	and crypto processing.
 * Initial autoconfiguration. Ability to extend it with additional
 	attributes if needed.
 * Support for any kind of network media (not limited to tcp or inet
 	protocols) higher MAC layer (socket layer). Out of the box
	kernel-side IPv6 support (needs to extend configuration utility,
	check how it was done in POHMELFS [1]).
 * Security attributes for local export nodes (list of allowed to
 	connect addresses with permissions).
 * Ability to use any supported cryptographically strong checksums.
 	Ability to encrypt data channel.

This release brings following changes:
 * Permission checks in export node. Read-only connections.
 * Remove DST node from the global table not only when it is freed, but
	also on demand with node del command.

I belive project is completed. Consider for inclusion.

One can grab sources (various configuration examples can be found
in 'userspace' dir) from archive, or via kernel and userspace GIT trees.

1. POHMELFS homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=pohmelfs

2. DST homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=dst

3. DST archive.
http://tservice.net.ru/~s0mbre/archive/dst/

4. DST git ...
From: Sven Wegener
Date: Monday, September 8, 2008 - 10:19 am

Some warnings:

  CC [M]  drivers/block/dst/state.o
drivers/block/dst/state.c: In function 'dst_recv_bio':
drivers/block/dst/state.c:396: warning: format '%llu' expects type 'long long unsigned int', but argument 3 has type 'sector_t'
drivers/block/dst/state.c: In function 'dst_process_io_response':
drivers/block/dst/state.c:434: warning: format '%llu' expects type 'long long unsigned int', but argument 3 has type 'sector_t'
  CC [M]  drivers/block/dst/export.o
drivers/block/dst/export.c: In function 'dst_accept':
drivers/block/dst/export.c:249: warning: 'main' is usually a function
drivers/block/dst/export.c: In function 'dst_export_read_request':
drivers/block/dst/export.c:405: warning: format '%llu' expects type 'long long unsigned int', but argument 3 has type 'sector_t'
drivers/block/dst/export.c: In function 'dst_process_io':
drivers/block/dst/export.c:520: warning: format '%llu' expects type 'long long unsigned int', but argument 3 has type 'sector_t'
drivers/block/dst/export.c: In function 'dst_export_send_bio':
drivers/block/dst/export.c:541: warning: format '%llu' expects type 'long long unsigned int', but argument 4 has type 'sector_t'
  CC [M]  drivers/block/dst/crypto.o
drivers/block/dst/crypto.c: In function 'dst_export_crypto_action':
drivers/block/dst/crypto.c:623: warning: format '%llu' expects type 'long long unsigned int', but argument 5 has type 'sector_t'
  CC [M]  drivers/block/dst/trans.o
drivers/block/dst/trans.c: In function 'dst_trans_insert':
drivers/block/dst/trans.c:87: warning: format '%llu' expects type 'long long unsigned int', but argument 4 has type 'sector_t'
drivers/block/dst/trans.c:87: warning: format '%llu' expects type 'long long unsigned int', but argument 8 has type 'sector_t'
drivers/block/dst/trans.c:95: warning: format '%llu' expects type 'long long unsigned int', but argument 4 has type 'sector_t'
drivers/block/dst/trans.c: In function 'dst_process_bio':
drivers/block/dst/trans.c:160: warning: format '%llu' expects type 'long long ...
From: Evgeniy Polyakov
Date: Monday, September 8, 2008 - 10:43 am

Hi Sven.

Thanks for the review.


Yup, sector_t is diffrent depending on arch and config options (u64 vs unsigned long),
so it is not possible to represent it correctly without dereferencing

In case of extended functionality there will be no need to change config

Debug prints are not supposed to be turned on except on very serious
conditions. In this case we do need to have as much info as possible (like


No, I do belive that grepping over |struct ablkcipher_request| when
something is about to be changed is more convenient, than searching for
the structure name and then object name itself.


I guess so. Commented code was used for older crypto initialization and


I just want to shut up the compiler, since failing to register
informational attribute is not critical. But it could also be



It was just copy-pasted :)

-- 
	Evgeniy Polyakov
--

From: Sven Wegener
Date: Monday, September 8, 2008 - 11:19 am

If there is coming more funtionality, ok, but currently it's irritating to 

Yeah, the dprintk()s are ok, but I was after the printk()s that follow. 

Guess I picked a bad example... There are some in the code that are 

Yeah, but in nearly all cases you don't need to change anything in that 
line, when you use sizeof(*ptr), because it always gets you the correct 
size you want. There are cases we could argue about, because you might 
want to do a limited memset() or memcpy() with a different size, but for 
most cases like memset() to zero or kmalloc, sizeof(*ptr) makes it clear 
that you want to clear out the whole structure or you want enough memory 
to store the type of ptr in it. Don't know how strict the policy is, but 

For just shutting up the compiler, use the void cast. I think removing a 

In other cases you allocate u8 iv[32] on the stack, so I guess this point 
is void. And I think 36 bytes isn't that much and it looks like you're 
called from user space, so I don't think you have a deep callchain here.
--

From: Evgeniy Polyakov
Date: Monday, September 8, 2008 - 12:31 pm

Nothing is actually planned, I do not care which config option will be
used for that, just do not want to get a new claims on that it should be



If I will ut (void) casting there, I'm pretty sure I will be blamed

That's a fast path, so I optimized a bit, but I agree, iv initializaion
could be also moved to the preallocated area.
I think connector message can be allocated on stack, since I know the
whole callchain in that path.

I will updated the patch.

-- 
	Evgeniy Polyakov
--

Previous thread: Controlling backlight on thinkpad x60 by Pavel Machek on Monday, September 8, 2008 - 8:10 am. (7 messages)

Next thread: [PATCH] x86: io-apic - do not use KERN_DEBUG marker too much by Cyrill Gorcunov on Monday, September 8, 2008 - 8:38 am. (6 messages)