login
Search
Hello everybody, I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library : dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size, PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0); Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose. Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ? Best regards Philippe --
Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page: http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html Cheers, Michael --
Interesting (and somewhat off topic to your conversation here) - it appears that when dumping mappings, the kernel ignores the maximum core size set with "limit". This is particularly interesting on a 64 bit kernel where a bug in your code causes you to try to read something about 2Gb into your alleged mmaped file (actual size ~500 bytes) and the segfault causes a coredump. Bron. --
Do you have a ssimple example program for this? -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html --
Hi Michael,
thanks for the info. I didn't know about /proc/PID/coredump_filter.
that part was promising :
bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.
The default value of coredump_filter is 0x3; this reflects traditional
Linux behavior and means that only anonymous memory segments are dumped.
Unfortunately, the part that applies to me (I have tested it) is the next one :
Memory-mapped I/O pages such as frame buffer are never dumped, [...],
regardless of the coredump_filter value.
Is that a design decision, or a mere finding of the way it is implemented
now ?
So, back to my original question :
Can some kind soul provide me some pointers to the way I should modify
the kernel to make the inclusion of the video1394 mmapped segment in
core files possible ?
best regards
Philippe
--
[CC+= hidehiro.kawai.ez@hitachi.com] Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight. Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html --
[CC+= linux1394-devel@lists.sourceforge.net] -- Philippe De Muyter phdm at macqel dot be Tel +32 27029044 Macq Electronique SA rue de l'Aeronef 2 B-1140 Bruxelles Fax +32 27029077 --
This shouldn't be a problem to you as far as I understand. I suppose "memory mapped I/O pages" means registers of PCI devices, mapped into the memory address space. The DMA buffer which you would like to get included in the core file is normal RAM (I suppose: allocated by the kernel in the kernel's virtual address space, mapped into the application client's address space by mmap(), and also mapped into the FireWire controller's local bus address space for it to write received data into). FireWire controllers also have memory-mapped I/O regions (per OHCI: at least one region of at least 2048 bytes size). But your process never accesses them, only the kernel drivers do (to start and stop DMA programs and so on --- while the DMA programs and the DMA data buffers live in normal RAM). -- Stefan Richter -=====-==--- -=== ---=- http://arcgraph.de/sr/ --
I agree with your analysis, but the sentence takes as exemple 'frame buffer' which I don't think are registers. I have tested with a video1394 client, and I do not get the mmapped video memory in the core file, even with /proc/PID/coredump_filter set to 0xf. So, while I agree it seems technically feasible, it does not seem to be currently implemented :( Philippe --
Hi,
MMIO pages have been not dumped since a long time ago, and I didn't target
them for the coredump_filter feature because I thought it was generally
danger to read MMIO pages. As for frame buffer we would be able to
safely access to it, but there is no way to tell it from other MMIO
The following patch may help. To dump MMIO pages, set bit 5 of
coredump_filter.
$ echo 0x23 > /proc/<PID>/coredump_filter
Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
---
This patch is not intended to be merged to the upstream kernel
because the safeness of reading VM_IO mappings has not been
proved.
Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
===================================================================
--- linux-2.6.26-rc5-mm3.orig/fs/binfmt_elf.c
+++ linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
@@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
if (vma->vm_flags & VM_ALWAYSDUMP)
goto whole;
- /* Do not dump I/O mapped devices or special mappings */
- if (vma->vm_flags & (VM_IO | VM_RESERVED))
+#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
+
+ /* By default, do not dump memory mapped I/O mappings */
+ if (vma->vm_flags & VM_IO) {
+ if (FILTER(MMIO))
+ goto whole;
return 0;
+ }
-#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
+ /* Do not dump special mappings */
+ if (vma->vm_flags & VM_RESERVED)
+ return 0;
/* By default, dump shared memory if mapped from an anonymous file. */
if (vma->vm_flags & VM_SHARED) {
Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
===================================================================
--- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h
+++ linux-2.6.26-rc5-mm3/include/linux/sched.h
@@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
#define MMF_DUMP_MAPPED_PRIVATE 4
#define MMF_DUMP_MAPPED_SHARED 5
#define MMF_DUMP_ELF_HEADERS 6
+#define MMF_DUMP_MMIO 7
#define MMF_DUMP_FILTER_SHIFT MMF_DUMPABLE_BITS
-#define ...Hello Hidehiro,
Thanks for your patch, but it will not help here. Before applying it blindly
I asked myself if the mmapped region was tagged VM_IO, because it is actually
a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone.
Details :
drivers/ieee1394/video1394.c:
static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
{
[...]
return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
}
drivers/ieee1394/dma.c:
int dma_region_mmap(struct dma_region *dma, struct file *file,
struct vm_area_struct *vma)
{
[...]
vma->vm_ops = &dma_region_vm_ops;
vma->vm_private_data = dma;
vma->vm_file = file;
vma->vm_flags |= VM_RESERVED;
return 0;
}
So, actually the zone I would like to get dumped in the core file is tagged
VM_RESERVED.
I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.
As I don't know the real meaning of VM_RESERVED, I do not know which choice
is the best one for the official kernel tree, but locally I'll go for
adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
Philippe
--
Hello, I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either. Allowing everyone to choose whether dump the dma region or not, perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and a coredump_filter bit which controls (VM_IO | VM_RESERVED) && VM_DUMPABLE area, for example. I think it is also OK to just add VM_ALWAYSDUMP flag to the dma region if the device driver knows the region is safely readable and small enough. Regards, -- Hidehiro Kawai Hitachi, Systems Development Laboratory Linux Technology Center --
I don't know these things either. But among else, VM_RESERVED prevents a vma from being swapped out. Makes kind of sense, given that besides It is safely readable. I don't know if it is small enough. The size of the DMA buffer is AFAIK chosen by userspace (by the application program or maybe a library) which uses the character device file ABIs for isochronous FireWire IO of raw1394, video1394, or dv1394. -- Stefan Richter -=====-==--- -=== --=-- http://arcgraph.de/sr/ --
For video1394, the size of the buffer is N times the size of a frame, with N choosen by the application but small (let say < 32). The size of the frame depends on the camera. I have already worked with 8bpp 1600x1200 cameras,but with a smaller N in that case. 1600x1200x32 gives 60M. If they were allocated by malloc instead of by mmap, they would be dumped in the core files without any question about the size. So I think we may consider the 'small enough' criteria is met. Best regards Philippe --
They're rather an embarrassment, nobody can say quite what VM_RESERVED and VM_IO really mean, beyond not-your-ordinary-vma: VM_VOODOO, beware. They both say "count me as reserved_vm", which means /proc/<pid>/status leaves them out of VmSize. Certainly VM_IO hints at special I/O memory, but in most cases there's nothing very special about either: just memory preallocated by a driver. In 2.4 VM_RESERVED indeed prevented swapout from looking at the pages of that vma. We probably ought to have killed it as part of the 2.5 switch to rmap. In 2.6 the pages of a VM_RESERVED vma wouldn't be considered for swapout, because they're not put on any LRU for that. I think I promised (ah, that's a shameful phrase!) to get rid of it a few years ago, but never quite got around to it. VM_RESERVED and VM_IO are treated as equivalent in most tests, but there's just a few places, e.g. get_user_pages, where we're scared off by VM_IO but don't mind VM_RESERVED. VM_ALWAYSDUMP: marvellous, a flag with an understandable name and a clear purpose. Hugh --
Hi Hidehiro, On Fri, Jul 04, 2008 at 02:50:25PM +0900, Hidehiro Kawai wrote: I have just submitted a patch doing that for ieee1394 dma regions: http://marc.info/?l=linux-kernel&m=121510404729225&w=2 Thanks for your help Philippe --
[Sorry, resent with CC += linux1394-devel@lists.sourceforge.net]
Hello Hidehiro,
Thanks for your patch, but it will not help here. Before applying it blindly
I asked myself if the mmapped region was tagged VM_IO, because it is actually
a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone.
Details :
drivers/ieee1394/video1394.c:
static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
{
[...]
return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
}
drivers/ieee1394/dma.c:
int dma_region_mmap(struct dma_region *dma, struct file *file,
struct vm_area_struct *vma)
{
[...]
vma->vm_ops = &dma_region_vm_ops;
vma->vm_private_data = dma;
vma->vm_file = file;
vma->vm_flags |= VM_RESERVED;
return 0;
}
So, actually the zone I would like to get dumped in the core file is tagged
VM_RESERVED.
I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.
As I don't know the real meaning of VM_RESERVED, I do not know which choice
is the best one for the official kernel tree, but locally I'll go for
adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
Philippe
--
Currently, core files do not contain the mmapped memory of the video1394 or dv1394 devices, which contain the actual video input, making it impossible to analyse the cause of abnormal program termination for image analysis or (de)compression software. Fix that. Signed-off-by: Philippe De Muyter <phdm@macqel.be> --- diff -r ced66ca0044f drivers/ieee1394/dma.c --- a/drivers/ieee1394/dma.c Mon Jun 30 08:58:09 2008 -0700 +++ b/drivers/ieee1394/dma.c Thu Jul 3 18:40:24 2008 +0200 @@ -274,7 +274,7 @@ int dma_region_mmap(struct dma_region *d vma->vm_ops = &dma_region_vm_ops; vma->vm_private_data = dma; vma->vm_file = file; - vma->vm_flags |= VM_RESERVED; + vma->vm_flags |= VM_RESERVED | VM_ALWAYSDUMP; return 0; } --
I'll commit it with the title changed to "dump mmapped iso buffers in core files" if that's OK. -- Stefan Richter -=====-==--- -=== --=-- http://arcgraph.de/sr/ --
It's an implementation issue, and there are patches queue for 2.6.27 that allow gdb access to these regions, which should be easily extendable to cover core dumps, too. --
[ sorry, reposted to add cc linux1394-devel@lists.sourceforge.net ] Hi Michael, thanks for the info. I didn't know about /proc/PID/coredump_filter. that part was promising : bit 2 Dump file-backed private mappings. bit 3 Dump file-backed shared mappings. The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped. Unfortunately, the part that applies to me (I have tested it) is the next one : Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value. Is that a design decision, or a mere finding of the way it is implemented now ? So, back to my original question : Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ? best regards Philippe --
