Re: How to avoid data copies in a driver ?

Previous thread: troubleshooting/debugging hard locks by Lee Howard on Wednesday, May 14, 2008 - 12:27 pm. (6 messages)

Next thread: [PATCH 2.6.26-rc2] driver:Remove unused macro KERNEL_OFFSET by Pradeep Singh Rautela on Wednesday, May 14, 2008 - 1:10 pm. (1 message)
From: Francis Moreau
Date: Wednesday, May 14, 2008 - 12:54 pm

Hello,

I'd like to optimize my driver, which receives data through a fifo and gives
them to a user space application. In turns this application moves this data
into a file.

To avoid several useless copies, I'd like the application to pass to the driver
a  file descriptor (?) to  the driver and then the driver can directly move the
received data to that file.

Could anybody give me some example of such scheme ?

Thanks,
-- 
Francis
--

From: Lennart Sorensen
Date: Wednesday, May 14, 2008 - 1:02 pm

If the application memory mapped the file, would it be able to simply
pass a pointer to that mapped file as part of the call to the driver and
the driver would place the data directly at the requested location which
would then be directly to the file?

-- 
Len Sorensen
--

From: Francis Moreau
Date: Thursday, May 15, 2008 - 12:44 am

On Wed, May 14, 2008 at 10:02 PM, Lennart Sorensen

So I would need to map this pointer into the kernel space, then fill it, take
care of cache coherency, unmap the kernel pointer.

Do you have any example of that in the kernel tree ?

BTW, data are received in interrupt context. Is it safe to put them in mapped
memory  (can I have page fault ?) in this context ?

Thanks
-- 
Francis
--

From: Lennart Sorensen
Date: Thursday, May 15, 2008 - 6:59 am

I was asking if that would work.  Does the to_user and from_user work on
a pointer from user space if that pointer points at a memory mapped

Oh I thought you just wanted user space to be able to ask for a chunk of
data.  I wouldn't want to write to a file in an interrupt handler.
Compared to the disk I/O a memory copy sounds rather insignificant to
the overall process.

-- 
Len Sorensen
--

From: Francis Moreau
Date: Friday, May 16, 2008 - 1:17 pm

On Thu, May 15, 2008 at 3:59 PM, Lennart Sorensen

I would say yes but I'm not really confident.

-- 
Francis
--

From: linux-os (Dick Johnson)
Date: Wednesday, May 14, 2008 - 2:23 pm

You memory-map the data. Impliment mmap() in your driver.
You can also impliment poll()  { select() } so your
application knows when new data are available.

You cannot use a user-mode file-descriptor in the kernel.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (5588.29 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.
--

From: Francis Moreau
Date: Thursday, May 15, 2008 - 12:40 am

Hello,

On Wed, May 14, 2008 at 11:23 PM, linux-os (Dick Johnson)

Why not ?

I'm suprised because what I need doens't seem so uncommon, usually
devices send or
receive data to/from files. So a helper (system call ?) to achieve
that other than the basic
read/write seems needed, no ?

-- 
Francis
--

From: Arnd Hannemann
Date: Thursday, May 15, 2008 - 4:50 am

Usually devices send or receive just data, and they shouldn't care about
file format, filesystems, permissions and all this stuff...

Regards,
Arnd

--

From: Francis Moreau
Date: Thursday, May 15, 2008 - 5:08 am

The fact is that the data usually ends into a file.

thanks
-- 
Francis
--

From: linux-os (Dick Johnson)
Date: Thursday, May 15, 2008 - 6:16 am

The kernel is designed to perform services on behalf of
a caller. The kernel itself doesn't have a process context.
Therefore, a file-descriptor, which requires a process context
to mean anything, is not useful within the kernel unless the
kernel either uses your process context (which happens
efficiently when YOU call the kernel) or it steals one
from somebody else, which means their context gets trashed.
Note that every process is created with at file descriptors
0, 1, and 2. It's only the process context that keeps them
separate.

That said, you can create a kernel-mode task in your driver.
That task would have a context. However it wouldn't be YOUR
context, so you would need to signal it when data was
available, adding overhead and communicate with it from.
your user-space program context. It would share the same
data-space as your driver so it wouldn't need to copy.
However, such data would get copied into kernel buffers
by the I/O code so it's a waste anyway. You save one
copy, which you would do with memory-mapping, plus you
have the added communications overhead.

Using memory mapping as previously advised, lets you DMA
data directly to a user if your hardware does DMA. It also
would allow you to save one copy, even if you don't have
DMA capabilities. As far as copies are concerned, there are
many copies before your data actually gets to a disk platter.
Those copies (usually) occur when the kernel doesn't have
anything else to do.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (5588.29 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by ...
From: Jeremy Fitzhardinge
Date: Friday, May 16, 2008 - 1:10 am

It's fairly rare to have an application which requires moving data to 
file with absolutely no processing; normally there's at least a bit of 
massaging/parsing/etc.  If that's really what you want to do, maybe you 
can do it with splice?  I haven't looked at it at all, but the intention 
is that you can splice file descriptors together, so you can splice your 
device fd to a file fd and have it all just work...

Alternatively you could read() from your device into a mmaped file.  
That's a single copy from device to file, which is about the best you 
can do without going to heroic lengths.

Also, it really depends on your application.  Is it a high-bandwidth 
thing in which the copy is a huge cost?  Or do you want to eliminate the 
copies because it seems like a nice thing to do?


    J
--

From: Francis Moreau
Date: Friday, May 16, 2008 - 4:44 am

Hello,


Well, every cases where a client <-> server exchange files. I wouldn't call

yes but the kernel I'm working on (2.6.16) doens't have splice support.

But relay may sound a good idea, and the version I use has sendfile support.

The drawback of relay: the user application can receive a file from the kernel
but the app can't send a file to the kernel.


Don't know for now.

I think using sendfile with relay may be a good answer without me becoming

No it has a real cost. The application receives files which are usually larger
than 1Go.

Thanks
-- 
Francis
--

From: Pavel Machek
Date: Wednesday, May 21, 2008 - 3:41 am

Previous thread: troubleshooting/debugging hard locks by Lee Howard on Wednesday, May 14, 2008 - 12:27 pm. (6 messages)

Next thread: [PATCH 2.6.26-rc2] driver:Remove unused macro KERNEL_OFFSET by Pradeep Singh Rautela on Wednesday, May 14, 2008 - 1:10 pm. (1 message)