SLUB 0:1 SLAB (OOM during massive parallel kernel builds)

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Alexey Dobriyan
Date: Tuesday, October 23, 2007 - 11:16 am

Abstract:

Nasty OOM killings appear during massive parallel kernel builds with
SLUB, but not with SLAB. By nasty I mean, cc1 processes are killed --
object files in .ccache and tree are corrupted which makes me to
blow up them entirely.


Setup:

92G ext2 partition mounted with noatime. .ccache, master tree, object
trees, build logs residing on it. 2G of RAM, 1G of swap partition.
CFQ scheduler. Core 2 Duo E6400. x86_64 arch.


Workload:

build ~250 configs some fat, some not, in parallel from one source
using O= and save logs to separate dir. This is what I do regularly here.


Observations:

With SLAB this workload never went to OOM killer.
With SLUB and pretty much all debugging enabled, it finishes to the end
(albeit slowly).
With SLUB and no debugging, OOM killer kicks in.

After I notice it at evening, e.g ssh to box doesn't work -- new OOM killings
appear.


Below is dmesg from fresh boot,  OOM messages and .config. Please, give me a
clue on how to proceed from there. If needed I can prepare cut-down scripts
for reproducing and more precise description of setup.

================================================================================
[this is the first OOM]
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 142   Cold: hi:   62, btch:  15 usd:  53
CPU    1: Hot: hi:  186, btch:  31 usd: 159   Cold: hi:   62, btch:  15 usd:  54
Active:386969 inactive:66065 dirty:0 writeback:0 unstable:0
 free:3431 slab:11685 mapped:243 pagetables:14621 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:2008kB inactive:1172kB present:12744kB pages_scanned:6852 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5688kB min:5708kB low:7132kB high:8560kB active:1548100kB inactive:261040kB present:2051880kB pages_scanned:12798112 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 13*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 518*4kB 6*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5688kB
Swap cache: add 4908248, delete 4908248, find 699858/1120395, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
17925 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 188662 or a child
Killed process 25288 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 146   Cold: hi:   62, btch:  15 usd:  61
CPU    1: Hot: hi:  186, btch:  31 usd: 140   Cold: hi:   62, btch:  15 usd:  58
Active:306488 inactive:146569 dirty:0 writeback:0 unstable:0
 free:3415 slab:11631 mapped:2 pagetables:14611 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1596kB inactive:1576kB present:12744kB pages_scanned:14865 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5616kB min:5708kB low:7132kB high:8560kB active:1226588kB inactive:582396kB present:2051880kB pages_scanned:13650148 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 498*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5616kB
Swap cache: add 4908570, delete 4908570, find 699873/1120438, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9622 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 187901 or a child
Killed process 25289 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 147   Cold: hi:   62, btch:  15 usd:  56
CPU    1: Hot: hi:  186, btch:  31 usd: 179   Cold: hi:   62, btch:  15 usd:  27
Active:423015 inactive:30323 dirty:0 writeback:0 unstable:0
 free:3429 slab:11630 mapped:2 pagetables:14605 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1600kB inactive:1572kB present:12744kB pages_scanned:14993 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5672kB min:5708kB low:7132kB high:8560kB active:1690460kB inactive:119720kB present:2051880kB pages_scanned:27640275 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 500*4kB 11*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5672kB
Swap cache: add 4908633, delete 4908633, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9326 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 187140 or a child
Killed process 25291 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 159   Cold: hi:   62, btch:  15 usd:  39
CPU    1: Hot: hi:  186, btch:  31 usd: 179   Cold: hi:   62, btch:  15 usd:  57
Active:325742 inactive:127034 dirty:0 writeback:0 unstable:0
 free:3444 slab:11630 mapped:2 pagetables:14599 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1600kB inactive:1572kB present:12744kB pages_scanned:14993 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5732kB min:5708kB low:7132kB high:8560kB active:1301244kB inactive:506676kB present:2051880kB pages_scanned:3441512 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 509*4kB 14*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5732kB
Swap cache: add 4908693, delete 4908693, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9818 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 186379 or a child
Killed process 25292 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  29
CPU    1: Hot: hi:  186, btch:  31 usd: 161   Cold: hi:   62, btch:  15 usd:  61
Active:11064 inactive:441279 dirty:0 writeback:0 unstable:0
 free:3445 slab:11629 mapped:2 pagetables:14593 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1600kB inactive:1572kB present:12744kB pages_scanned:15057 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5736kB min:5708kB low:7132kB high:8560kB active:42656kB inactive:1763544kB present:2051880kB pages_scanned:4028526 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 520*4kB 9*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5736kB
Swap cache: add 4908753, delete 4908753, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10207 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 185618 or a child
Killed process 25293 (akmk)
as invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 151   Cold: hi:   62, btch:  15 usd:  57
CPU    1: Hot: hi:  186, btch:  31 usd: 165   Cold: hi:   62, btch:  15 usd:  47
Active:363161 inactive:89780 dirty:0 writeback:0 unstable:0
 free:3430 slab:11629 mapped:2 pagetables:14587 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1604kB inactive:1568kB present:12744kB pages_scanned:15185 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5676kB min:5708kB low:7132kB high:8560kB active:1453272kB inactive:355248kB present:2051880kB pages_scanned:37350819 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 511*4kB 6*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5676kB
Swap cache: add 4908822, delete 4908822, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9525 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 184857 or a child
Killed process 25297 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 148   Cold: hi:   62, btch:  15 usd:  21
CPU    1: Hot: hi:  186, btch:  31 usd: 180   Cold: hi:   62, btch:  15 usd:  57
Active:256272 inactive:196720 dirty:0 writeback:0 unstable:0
 free:3430 slab:11627 mapped:2 pagetables:14581 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1604kB inactive:1568kB present:12744kB pages_scanned:15249 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5676kB min:5708kB low:7132kB high:8560kB active:1023484kB inactive:785312kB present:2051880kB pages_scanned:4581896 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 513*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5676kB
Swap cache: add 4908882, delete 4908882, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9399 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 184096 or a child
Killed process 25302 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 159   Cold: hi:   62, btch:  15 usd:  44
CPU    1: Hot: hi:  186, btch:  31 usd: 188   Cold: hi:   62, btch:  15 usd:  57
Active:304839 inactive:148090 dirty:0 writeback:0 unstable:0
 free:3429 slab:11627 mapped:2 pagetables:14575 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1588kB inactive:1568kB present:12744kB pages_scanned:15441 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5672kB min:5708kB low:7132kB high:8560kB active:1215336kB inactive:593352kB present:2051880kB pages_scanned:13164262 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 512*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5672kB
Swap cache: add 4908968, delete 4908968, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9331 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 183335 or a child
Killed process 25304 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  30
CPU    1: Hot: hi:  186, btch:  31 usd: 165   Cold: hi:   62, btch:  15 usd:  53
Active:169953 inactive:282282 dirty:0 writeback:0 unstable:0
 free:3418 slab:11628 mapped:2 pagetables:14569 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1604kB inactive:1568kB present:12744kB pages_scanned:15505 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5628kB min:5708kB low:7132kB high:8560kB active:678208kB inactive:1127560kB present:2051880kB pages_scanned:4001527 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 498*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5616kB
Swap cache: add 4909029, delete 4909029, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10125 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 182574 or a child
Killed process 25305 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 180   Cold: hi:   62, btch:  15 usd:  53
CPU    1: Hot: hi:  186, btch:  31 usd: 164   Cold: hi:   62, btch:  15 usd:  59
Active:11285 inactive:440793 dirty:0 writeback:0 unstable:0
 free:3429 slab:11626 mapped:1 pagetables:14563 bounce:0
DMA free:8040kB min:32kB low:40kB high:48kB active:1724kB inactive:1428kB present:12744kB pages_scanned:7123 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5676kB min:5708kB low:7132kB high:8560kB active:45772kB inactive:1759440kB present:2051880kB pages_scanned:4827940 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 14*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8040kB
DMA32: 513*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5676kB
Swap cache: add 4909091, delete 4909091, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10098 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 181813 or a child
Killed process 25307 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 163   Cold: hi:   62, btch:  15 usd:  49
CPU    1: Hot: hi:  186, btch:  31 usd: 160   Cold: hi:   62, btch:  15 usd:  61
Active:140594 inactive:311525 dirty:0 writeback:0 unstable:0
 free:3430 slab:11625 mapped:1 pagetables:14558 bounce:0
DMA free:8040kB min:32kB low:40kB high:48kB active:1724kB inactive:1428kB present:12744kB pages_scanned:7315 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5680kB min:5708kB low:7132kB high:8560kB active:560652kB inactive:1244672kB present:2051880kB pages_scanned:21209367 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 14*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8040kB
DMA32: 514*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5680kB
Swap cache: add 4909153, delete 4909153, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10009 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 181052 or a child
Killed process 25308 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 172   Cold: hi:   62, btch:  15 usd:  59
Active:163601 inactive:288519 dirty:0 writeback:0 unstable:0
 free:3445 slab:11625 mapped:2 pagetables:14553 bounce:0
DMA free:8040kB min:32kB low:40kB high:48kB active:1724kB inactive:1428kB present:12744kB pages_scanned:7379 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5740kB min:5708kB low:7132kB high:8560kB active:652680kB inactive:1152648kB present:2051880kB pages_scanned:3581883 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 14*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8040kB
DMA32: 529*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5740kB
Swap cache: add 4909215, delete 4909215, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9926 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 180291 or a child
Killed process 25309 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 115   Cold: hi:   62, btch:  15 usd:  42
CPU    1: Hot: hi:  186, btch:  31 usd:  58   Cold: hi:   62, btch:  15 usd:  59
Active:8992 inactive:444596 dirty:0 writeback:0 unstable:0
 free:3422 slab:11588 mapped:1 pagetables:14545 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1556kB inactive:1248kB present:12744kB pages_scanned:14668 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5656kB min:5708kB low:7132kB high:8560kB active:34412kB inactive:1777136kB present:2051880kB pages_scanned:8923551 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 514*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5656kB
Swap cache: add 4931119, delete 4931119, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9791 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 179530 or a child
Killed process 25313 (akmk)
fixdep invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802e0c03>] prio_tree_insert+0x23/0x270
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8026b0f5>] __slab_free+0x1e5/0x330
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  14
CPU    1: Hot: hi:  186, btch:  31 usd:  72   Cold: hi:   62, btch:  15 usd:  30
Active:29737 inactive:423544 dirty:0 writeback:0 unstable:0
 free:3433 slab:11586 mapped:1 pagetables:14539 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1696kB inactive:1396kB present:12744kB pages_scanned:14830 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:117252kB inactive:1692780kB present:2051880kB pages_scanned:3553979 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931180, delete 4931180, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10157 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 178769 or a child
Killed process 25318 (akmk)
bash invoked oom-killer: gfp_mask=0x1280d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802593a5>] handle_mm_fault+0x485/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8026b0f5>] __slab_free+0x1e5/0x330
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  52
CPU    1: Hot: hi:  186, btch:  31 usd:  84   Cold: hi:   62, btch:  15 usd:  48
Active:190765 inactive:262200 dirty:0 writeback:0 unstable:0
 free:3424 slab:11586 mapped:2 pagetables:14533 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1696kB inactive:1396kB present:12744kB pages_scanned:15022 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:759060kB inactive:1049708kB present:2051880kB pages_scanned:7773323 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931241, delete 4931241, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10263 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 178008 or a child
Killed process 25321 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  35
CPU    1: Hot: hi:  186, btch:  31 usd:  94   Cold: hi:   62, btch:  15 usd:  50
Active:161540 inactive:291160 dirty:0 writeback:0 unstable:0
 free:3424 slab:11587 mapped:1 pagetables:14527 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1696kB inactive:1396kB present:12744kB pages_scanned:15086 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:644464kB inactive:1163132kB present:2051880kB pages_scanned:35575277 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 514*4kB 3*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931301, delete 4931301, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10530 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 177247 or a child
Killed process 25326 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  55
CPU    1: Hot: hi:  186, btch:  31 usd: 107   Cold: hi:   62, btch:  15 usd:  14
Active:40327 inactive:412498 dirty:0 writeback:0 unstable:0
 free:3433 slab:11587 mapped:1 pagetables:14521 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1684kB inactive:1392kB present:12744kB pages_scanned:15214 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:157320kB inactive:1650776kB present:2051880kB pages_scanned:3954392 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931361, delete 4931361, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10400 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 176486 or a child
Killed process 25331 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 113   Cold: hi:   62, btch:  15 usd:  47
CPU    1: Hot: hi:  186, btch:  31 usd: 119   Cold: hi:   62, btch:  15 usd:  49
Active:363535 inactive:88985 dirty:0 writeback:0 unstable:0
 free:3484 slab:11587 mapped:1 pagetables:14516 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1700kB inactive:1392kB present:12744kB pages_scanned:15214 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5904kB min:5708kB low:7132kB high:8560kB active:1452440kB inactive:354548kB present:2051880kB pages_scanned:4327701 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 566*4kB 7*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5904kB
Swap cache: add 4931422, delete 4931422, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10564 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 175725 or a child
Killed process 25339 (akmk)
fixdep invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802e0c03>] prio_tree_insert+0x23/0x270
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff802735be>] sys_newfstat+0x2e/0x50
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  50
CPU    1: Hot: hi:  186, btch:  31 usd: 129   Cold: hi:   62, btch:  15 usd:  43
Active:15013 inactive:437762 dirty:0 writeback:0 unstable:0
 free:3424 slab:11586 mapped:1 pagetables:14510 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1700kB inactive:1392kB present:12744kB pages_scanned:15278 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:58352kB inactive:1749656kB present:2051880kB pages_scanned:21495882 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931482, delete 4931482, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10239 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 174964 or a child
Killed process 25347 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff80282705>] dput+0xa5/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 112   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 142   Cold: hi:   62, btch:  15 usd:  58
Active:156024 inactive:296851 dirty:0 writeback:0 unstable:0
 free:3424 slab:11585 mapped:1 pagetables:14504 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1388kB present:12744kB pages_scanned:15470 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:624500kB inactive:1183968kB present:2051880kB pages_scanned:32711791 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931542, delete 4931542, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10070 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 174203 or a child
Killed process 25354 (akmk)
rm invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8026b0f5>] __slab_free+0x1e5/0x330
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 108   Cold: hi:   62, btch:  15 usd:  48
CPU    1: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  58
Active:382351 inactive:70856 dirty:0 writeback:0 unstable:0
 free:3424 slab:11584 mapped:1 pagetables:14499 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1388kB present:12744kB pages_scanned:15534 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:1527700kB inactive:282036kB present:2051880kB pages_scanned:6581042 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 504*4kB 8*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931603, delete 4931603, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9703 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 173442 or a child
Killed process 25369 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 118   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 162   Cold: hi:   62, btch:  15 usd:  20
Active:256144 inactive:196747 dirty:0 writeback:0 unstable:0
 free:3424 slab:11585 mapped:1 pagetables:14493 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1408kB present:12744kB pages_scanned:15598 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:1020568kB inactive:788012kB present:2051880kB pages_scanned:5734170 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 506*4kB 7*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931664, delete 4931664, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9917 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 172681 or a child
Killed process 25379 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff802735be>] sys_newfstat+0x2e/0x50
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 118   Cold: hi:   62, btch:  15 usd:  25
CPU    1: Hot: hi:  186, btch:  31 usd: 172   Cold: hi:   62, btch:  15 usd:  46
Active:423920 inactive:28829 dirty:0 writeback:0 unstable:0
 free:3418 slab:11584 mapped:6 pagetables:14487 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1436kB present:12744kB pages_scanned:15726 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5640kB min:5708kB low:7132kB high:8560kB active:1693976kB inactive:113880kB present:2051880kB pages_scanned:4012861 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 501*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5604kB
Swap cache: add 4931725, delete 4931725, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10104 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 171920 or a child
Killed process 25388 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 117   Cold: hi:   62, btch:  15 usd:  56
CPU    1: Hot: hi:  186, btch:  31 usd: 181   Cold: hi:   62, btch:  15 usd:  36
Active:433664 inactive:18732 dirty:0 writeback:0 unstable:0
 free:3439 slab:11583 mapped:3 pagetables:14483 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:15726 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5724kB min:5708kB low:7132kB high:8560kB active:1732948kB inactive:73496kB present:2051880kB pages_scanned:5885606 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 513*4kB 11*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5724kB
Swap cache: add 4931785, delete 4931785, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10317 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 171159 or a child
Killed process 25395 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 136   Cold: hi:   62, btch:  15 usd:  58
CPU    1: Hot: hi:  186, btch:  31 usd: 160   Cold: hi:   62, btch:  15 usd:  44
Active:191270 inactive:261290 dirty:0 writeback:0 unstable:0
 free:3425 slab:11584 mapped:3 pagetables:14477 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:15918 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5668kB min:5708kB low:7132kB high:8560kB active:763372kB inactive:1043728kB present:2051880kB pages_scanned:31767483 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 517*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5668kB
Swap cache: add 4931845, delete 4931845, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10106 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 170398 or a child
Killed process 25397 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 135   Cold: hi:   62, btch:  15 usd:  15
CPU    1: Hot: hi:  186, btch:  31 usd: 171   Cold: hi:   62, btch:  15 usd:  44
Active:162477 inactive:289899 dirty:0 writeback:0 unstable:0
 free:3419 slab:11584 mapped:4 pagetables:14471 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:15982 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5644kB min:5708kB low:7132kB high:8560kB active:648200kB inactive:1158164kB present:2051880kB pages_scanned:3195902 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 502*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5608kB
Swap cache: add 4931906, delete 4931906, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10326 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 169637 or a child
Killed process 25401 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 140   Cold: hi:   62, btch:  15 usd:  57
CPU    1: Hot: hi:  186, btch:  31 usd: 183   Cold: hi:   62, btch:  15 usd:  57
Active:166888 inactive:285231 dirty:0 writeback:0 unstable:0
 free:3440 slab:11584 mapped:4 pagetables:14465 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:16046 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5728kB min:5708kB low:7132kB high:8560kB active:665844kB inactive:1139492kB present:2051880kB pages_scanned:3455232 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 532*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5728kB
Swap cache: add 4931966, delete 4931966, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10416 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 168876 or a child
Killed process 25407 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 158   Cold: hi:   62, btch:  15 usd:  17
CPU    1: Hot: hi:  186, btch:  31 usd: 182   Cold: hi:   62, btch:  15 usd:  48
Active:8518 inactive:443658 dirty:0 writeback:0 unstable:0
 free:3425 slab:11584 mapped:4 pagetables:14459 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:16110 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5668kB min:5708kB low:7132kB high:8560kB active:34348kB inactive:1771152kB present:2051880kB pages_scanned:18797855 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 511*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5668kB
Swap cache: add 4932027, delete 4932027, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10343 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 168115 or a child
Killed process 25409 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 156   Cold: hi:   62, btch:  15 usd:  57
CPU    1: Hot: hi:  186, btch:  31 usd: 184   Cold: hi:   62, btch:  15 usd:  47
Active:168463 inactive:283970 dirty:0 writeback:0 unstable:0
 free:3426 slab:11581 mapped:4 pagetables:14453 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1712kB inactive:1428kB present:12744kB pages_scanned:16238 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5672kB min:5708kB low:7132kB high:8560kB active:672140kB inactive:1134452kB present:2051880kB pages_scanned:10945764 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 518*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5672kB
Swap cache: add 4932088, delete 4932088, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9997 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 167354 or a child
Killed process 25418 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025d4d1>] vma_merge+0x161/0x1f0
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 154   Cold: hi:   62, btch:  15 usd:  47
CPU    1: Hot: hi:  186, btch:  31 usd: 163   Cold: hi:   62, btch:  15 usd:  47
Active:336342 inactive:115515 dirty:0 writeback:0 unstable:0
 free:3445 slab:11581 mapped:1 pagetables:14447 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:2096kB inactive:996kB present:12744kB pages_scanned:9433 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5736kB min:5708kB low:7132kB high:8560kB active:1343272kB inactive:461064kB present:2051880kB pages_scanned:3723650 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 9*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 524*4kB 7*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5736kB
Swap cache: add 4932148, delete 4932148, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10461 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 166593 or a child
Killed process 25428 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 167   Cold: hi:   62, btch:  15 usd:  54
CPU    1: Hot: hi:  186, btch:  31 usd: 123   Cold: hi:   62, btch:  15 usd:  52
Active:2906 inactive:450980 dirty:0 writeback:0 unstable:0
 free:3419 slab:11579 mapped:16 pagetables:14474 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1752kB inactive:1396kB present:12744kB pages_scanned:11565 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5640kB min:5708kB low:7132kB high:8560kB active:9872kB inactive:1802524kB present:2051880kB pages_scanned:17405264 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 514*4kB 2*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5640kB
Swap cache: add 4936171, delete 4936171, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9649 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 165832 or a child
Killed process 25438 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 164   Cold: hi:   62, btch:  15 usd:  42
CPU    1: Hot: hi:  186, btch:  31 usd: 136   Cold: hi:   62, btch:  15 usd:  54
Active:448616 inactive:4562 dirty:0 writeback:0 unstable:0
 free:3421 slab:11578 mapped:16 pagetables:14468 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1752kB inactive:1396kB present:12744kB pages_scanned:11565 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5648kB min:5708kB low:7132kB high:8560kB active:1792712kB inactive:16852kB present:2051880kB pages_scanned:10911644 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 512*4kB 4*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Alexey Dobriyan, (Tue Oct 23, 11:16 am)
Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Christoph Lameter, (Tue Oct 23, 12:35 pm)
Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Christoph Lameter, (Tue Oct 23, 12:57 pm)
Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Christoph Lameter, (Tue Oct 23, 1:09 pm)
Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Christoph Lameter, (Wed Oct 24, 9:56 am)
Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Alexey Dobriyan, (Wed Oct 24, 12:28 pm)
Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Christoph Lameter, (Wed Oct 24, 7:15 pm)
Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds), Christoph Lameter, (Wed Oct 24, 7:43 pm)