login
Search
I am preparing a server for data acquisition from a big CCD camera. It uses PCI controllers which came with some software including a kernel driver module for 32-bit systems. Specs say Fedora 8 (2.6.26 kernel) but I found it working also in Centos 5.3 (2.6.18 PAE kernel). The driver requires that the appropriate amount of RAM is reserved for its use and it seems that it MUST be under 4GB. So I use, as instructed:
kernel /boot/vmlinuz-2.6.18-128.el5PAE ro root=LABEL=/ rhgb quiet mem=2768175104 memmap=587202560$2768175104
line in grub.conf. But that results in all memory above 4GB being "lost", wasted. The server is a powerful, 16GB RAM machine and the images will be huge, 512MB each (256MB from each of the two controllers) so the more usable memory to process the images, the better.
Thinkable solutions I am looking the advice for are:
1. configuring the kernel so that it reserves a "hole" in the RAM, still being able to use the "upper" memory
1a. as 1. but, if it is impossible to use the upper memory in a "regular" way, at least being able to setup a RAM disk there
2. forcing the kernel driver to use the memory above 4GB
3. rewriting the kernel module to 64-bit
Options 2 and 3 require kernel modules programming skills which I do not have. I am pretty fluent in C but have never tried kernel programming.
Any hints would be very much appreciated.
regards, Michal.
why mem=
do you really have to use mem= to limit the memory to the region below the hole? the documentation for memmap= in Documentation/kernel-parameters.txt says memmap=...$... marks a region of memory as reserved. if you had to use it together with mem=, it would essentially reserve all the memory after that and there would be no need for the size parameter.
i read the 2nd paragraph about mem= that you have to use memmap= if you use mem=, not the other way round.
I tried to reply but the
I tried to reply but the post was considered a spam. Complete absurd.
Is it possible to check if
Is it possible to check if the memory specified with memmap is really reserved? With mem option, dmesg shows user memory map at boot, but without mem, no trace of reserving anything can be found.
another problem w/o map option
Getting the data stored in the buffer is pretty slow when I use the supplied software. The software library "opens" the controller, maps the RAM buffer for its own purposes and returns the pointer to the mapped virtual(?) memory. When I run a small app just opening the controller and then 'memcpy' from this mapped pointer to a buffer allocated by this app (by regular malloc), it takes 22 secs to copy 520MB (23MB/s). I am not sure why is it so, I guess only that the data goes again through the controller. So, I tried to access the RAM buffer directly, using (as root, of course):
fd = open("/dev/mem", O_RDONLY);
buf = mmap(NULL, len, PROT_READ, MAP_SHARED, fd, ram);
where 'ram' is the physical address as in memmap=
From this 'buf' I can then memcpy 520MB in a fraction of a second, as I would expect from copying memory to memory. The problem is that this trick works when the memory is "unused" (with mem=ADDR option) but does not work when I use the memmap= option only. 'mmap' fails with "Operation not permitted". Is there way to get direct access to that memory?