Hi list,
i'm developing an application (in C) which needs to write about

1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI.

It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. 
I buffer the data in (currently 4) 400Mb buffers and use write(2) in a

dedicated thread to write them to the raw disk (no fs). 
The write(2) performance is not good enough, the writer threads take to

much time, and i ask you for ideas, howto to boost the write

performance. 
Maybe mmaping the disk would work? 
Cheers,
-Michael
PS. I would like to be cc'd as i usually don't read the list due to high

traffic.
-

People regularly report speeds higher than that on the RAID list, and I

can get that order of magnitude speed using dd with 1MB buffers to a

software RAID-0 array or cheap SATA drives. Are you using a decent

controller? Many "RAID" controllers have bandwidth limitations, buffer

size issues, etc. I had some chea "SCSI" arrays which were just SCSI 
I would limit the write size to a MB and see if that helps, regardless

of the buffer size. A circular queue of smaller buffers, like ptbuf, may 
I don't think it would help, I'd really try limiting the size of the

write() calls first, assuming your hardware is adequate.
--

Bill Davidsen <davidsen@tmr.com>

   "We have more to fear from the bungling of the incompetent than from

the machinations of the wicked."  - from Slashdot

-

The kernel really cannot sustain 125MB/s? I assume the disk

array is capable?
Where is the bottleneck? Does it keep all disks busy, or are

the CPUs overloaded?

-

Yes, the array should be capable, LSI controller U320, SATA disks inside
I'm not sure where the bottleneck is. CPU Load goes up with

many pdflush processes in D state.
Thanks for the suggestions with sg and O_DIRECT so far,

will try this.
-Michael
-

You could use an O_DIRECT write if the data is suitably aligned

and your IO sizes are big enough (O_DIRECT is usually a loss

on small IOs). It will also be synchronous, but if you do it

from a separate thread anyways that should be fine.
-Andi

-

Thanks to all who answered and especially to Andi. Using O_DIRECT did

the trick.
Cheers,
Michael
-

If your target is a SCSI target you can gain up to 15% by using

sg. Search on the net for the "sg utils" package. The source code

of sg_dd and others are a grate example of how to do it.
Other wise O_DIRECT is your friend. Also look for asynchronous

I/O so you have a few pending IO buffers to keep the pipes full.
Boaz
-

Boaz Harrosh wrote:
What's the advantage of sg_io over O_DIRECT to the block device?

I think sg devices are being (slowly?) obsoleted since most stuff

which has been in sg now works over normal block device, no?

At least, I haven't seen any significant difference between

sg_dd and dd with oflag=direct.
Thanks.
/mjt

-

El Tue, 09 Oct 2007 15:50:17 +0200
Create a 1GB ram disk, write data there, and then backup into a hard

disk
--

Gustavo Chaín Dumit

-