[Follow-up] Physical memory disappeared from /proc/meminfo

Previous thread: 2.6.27-rc3-git4: Multiple implicit declarations in log2.h by Rafael J. Wysocki on Sunday, August 17, 2008 - 10:32 am. (1 message)

Next thread: pull request: wireless-2.6 2008-08-17 by John W. Linville on Sunday, August 17, 2008 - 11:56 am. (9 messages)
From: Marc Villemade
Date: Sunday, August 17, 2008 - 10:59 am

Hi everyone,

(I apologize in advance for this long email)

While looking for answers to my memory problems i've been having for
some time now, i've stumbled onto these posts:

Dated last year :
http://kerneltrap.org/mailarchive/linux-kernel/2007/8/26/164909

and dated from a couple months ago:
http://kerneltrap.org/mailarchive/linux-kernel/2008/6/24/2209554

I'm having exactly the same issue but on a 2.6.20.4 vanilla kernel
(x86). /proc/meminfo shows that
MemFree+Buffers+cached+AnonPages+Slab+Mapped != MemTotal, which AFAIK
should be the case.

6_days_uptime_machine ~ # cat /proc/meminfo
MemTotal:      3106668 kB
MemFree:        678104 kB
Buffers:        120024 kB
Cached:          69892 kB
SwapCached:          0 kB
Active:         740872 kB
Inactive:      1621704 kB
HighTotal:     2227996 kB
HighFree:        21380 kB
LowTotal:       878672 kB
LowFree:        656724 kB
SwapTotal:     4192956 kB
SwapFree:      4192956 kB
Dirty:            1292 kB
Writeback:           0 kB
AnonPages:      586900 kB
Mapped:          13824 kB
Slab:            50432 kB
SReclaimable:    39092 kB
SUnreclaim:      11340 kB
PageTables:       1532 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   5746288 kB
Committed_AS:  1073624 kB
VmallocTotal:   114680 kB
VmallocUsed:      8944 kB
VmallocChunk:   105568 kB

in contrast, here's the meminfo from a machine that has been rebooted
20 hours before, in which the above mentioned figures almost add up to
MemTotal. There's 50 MB missing already which makes me think the leak
starts right away after boot up...

20_hours_uptime_machine ~ # cat /proc/meminfo
MemTotal:      3106668 kB
MemFree:       2455932 kB
Buffers:         88624 kB
Cached:          69364 kB
SwapCached:          0 kB
Active:         496772 kB
Inactive:       114680 kB
HighTotal:     2227996 kB
HighFree:      1695240 kB
LowTotal:       878672 kB
LowFree:        760692 kB
SwapTotal:     4192956 kB
SwapFree:      4192956 kB
Dirty:            ...
From: Rik van Riel
Date: Sunday, August 17, 2008 - 11:59 am

On Sun, 17 Aug 2008 19:59:10 +0200

There are also page tables, vmalloc memory and various
unaccounted kernel allocations.

-- 
All rights reversed.
--

From: Marc Villemade
Date: Sunday, August 17, 2008 - 12:26 pm

hey

Thanks for your answer, but it still doesn't come close. I'm up to 1.5
Gig adding what i said. and PageTables + VMallocTotal equals 160 Mo.
Sounds unlikely to me that something unaccounted for might go up to
1.4 Gig, don't you think ? Or maybe it is accounted somewhere else
than /proc/meminfo ? Please point me to the right place if so.

Cheers

Marc Villemade


--

From: Arjan van de Ven
Date: Sunday, August 17, 2008 - 12:42 pm

On Sun, 17 Aug 2008 21:26:07 +0200

you're not accidentally using shmfs or tmpfs are you ?
(or otherwise have a leak in sysv shm)


you can use the "ipcs" command to see your shm allocation

(and for tmpfs/shmfs... check if you have it mounted anywhere.
Something like /dev being on shmfs and /dev/null being a file not a
device can.. totally screw you over)
--

From: Marc Villemade
Date: Sunday, August 17, 2008 - 1:22 pm

hey,

here are my mount points:

6_days_uptime_machine ~ # mount
/dev/evms/root on / type ext3 (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,nosuid)
devpts on /dev/pts type devpts (rw,nosuid,noexec)
/dev/evms/boot on /boot type ext3 (rw,noatime)
/dev/evms/data on /data type ext3 (rw,noatime,data=journal)
/dev/evms/queue on /mnt/queue type ext3 (rw,noatime)
none on /dev/shm type tmpfs (rw)
/dev on /opt/bizanga/chroot/dev type none (rw,bind)
usbfs on /proc/bus/usb type usbfs (rw,noexec,nosuid,devmode=0664,devgid=85)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc
(rw,noexec,nosuid,nodev)

so shmfs is never used, but tmpfs is for /dev and /dev/shm. Do you
think that would be an issue ?

and it looks like i'm not using any shared mem either :
6_days_uptime_machine ~ # ipcs -a

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status

------ Semaphore Arrays --------
key        semid      owner      perms      nsems

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages


And i've also checked that all /dev entries are devices. Thanks for the tip.

After Ryk's email, i started looking a little more closely to the
other values in /proc/meminfo and /proc/zoneinfo and i would like to
have more information about zoneinfo. It seems that most of the
inactive pages are in HighMem :

        active   148512
        inactive 403735
        scanned  0 (a: 0 i: 0)

and what does the scanned line mean ? And what about the fact that
it's all 0 ? Means that it's never been scanned for reclaimable space
? Maybe that's where my problem lies !? Any more info on those proc
entries would be greatly appreciated. My friend google didn't have a
lot of good answers, neither did the Kernel documentation.


Marc Villemade
--

From: Andi Kleen
Date: Sunday, August 17, 2008 - 11:09 pm

Nope, the equation is not necessarily true. 
See http://halobates.de/memorywaste.pdf for a detailed discussion.

-Andi
--

Previous thread: 2.6.27-rc3-git4: Multiple implicit declarations in log2.h by Rafael J. Wysocki on Sunday, August 17, 2008 - 10:32 am. (1 message)

Next thread: pull request: wireless-2.6 2008-08-17 by John W. Linville on Sunday, August 17, 2008 - 11:56 am. (9 messages)