On Thu, Sep 13, 2007 at 05:25:32PM -0300, Alejandro Lozanoff wrote:

quoted text
> Hello list,

re
> We recently updated a 3.7 machine running awstat(perl) to parse all our

quoted text
> websites logs with the biggest being around 1GB.

> When parsing the big log it randomly segfaults on 4.1, 3.9 and 3.8, we

> tried new clean release installs and it still segfaults. On 3.7 it works

> flawlessly, on 3.8 which has the same perl version as 3.7 (5.8.6) it

> still segfaults. The problem is completely random but it tends to happen

> after its been running for a while as it doesnt happen on small logs (or

> the probability for it to happen on those files is too low )

>

> As the trace below (from the perl.core) shows, it's an out of bounds

> problem, then we remember about a change on 3.8:

>

> "malloc(3) has been rewritten to use the mmap(2) system call,

> introducing unpredictable allocation addresses and guard pages, which

> helps in detecting heap based buffer overflows and prevents various

> types of attacks."

yes. this increases memory fragmentation immensly resulting

in (practically) less virtual space available for data.

as an increased penalty (200-300%) for cpu consuption

on processes w/ lots (20M and more) malloc(3)ed memory...

as well increased demand for the physical memory that

on the overcommiting nature of it you perhaps observe.

a way around it is only to use perl malloc (sbrk-based)
cu

--

    paranoic mickey       (my employers have changed but, the name has remained)