GNOME applications use a Python driven XML parser to generate help

document translations.  The engine takes the English XML document and

applies translations to it.  The tool, xml2po, is installed as part of

textproc/gnome-doc-utils.  I'm currently working on porting GNOME 2.21,

and one of the Evolution help doc changes triggered a memory problem on

my test machine.  Basically, with up to and including rev 1.154 of

malloc.c, I am able to generate the help file with no errors.  With all

later revs including 1.160, the Python process balloons up to about 512

MB of memory, then dies.
The only malloc config I've done is symlink Aj to /etc/malloc.conf.  I'd

be happy to provide the file an exact command used, but it might be

easier to let me know if there's any debugging I could provide that

would help here.
Joe
--=20

Joe Marcus Clarke

FreeBSD GNOME Team      ::      gnome@FreeBSD.org

FreeNode / #freebsd-gnome

http://www.FreeBSD.org/gnome

What is the state of the PYMALLOC option presented when building python

from ports? Do a make config to toggle it.

Revision 1.160 of malloc.c uses sbrk instead of mmap by default.  The

intent is to make data segment resource limits useful.  So, there are

two possible explanations I can think of for your problems.  The first

possibility is that there is still a bug in malloc that you're hitting.

  However, the other possibility is that the python program you're

running really needs more than 512 MB of memory, and you're hitting the

resource limit.
It would be really helpful to me if you run your program with

MALLOC_OPTIONS=dM and monitor memory usage.  These flags cause mmap to

be used instead of sbrk, and we can find out from that how much memory

you really need.  If peak memory usage is substantially different when

using mmap versus sbrk, there's probably a malloc bug.
This latest round of malloc changes hasn't been much fun.  Thanks for

your help and patience.
Jason

_______________________________________________

freebsd-current@freebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-current

To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"

Memory climbed up to 976 MB SZ, 974 MB RSS MB with dM

-> /etc/malloc.conf.  The file was eventually generated without error.

Again, with Aj -> /etc/malloc.conf, the python2.5 process operating on
No problem.  Thanks for getting back to me.
Joe
--=20

Joe Marcus Clarke

FreeBSD GNOME Team      ::      gnome@FreeBSD.org

FreeNode / #freebsd-gnome

http://www.FreeBSD.org/gnome

Okay, that indicates that there is not a problem with malloc; you're

running into the data segment resource limit.  It isn't possible to

increase the data segment beyond 512 MB on i386, so your best bet is to

use MALLOC_OPTIONS=DM for the memory-intensive program.  That will cause

the program use all available space in the data segment, then start

using mmap as necessary.
I'm sorta thinking that MALLOC_OPTIONS=DM should be the default.  Robert

Watson is the person who talked me into this change, so feel free to

give him a hard time about the extra configuration you have to do in

order to get work done. =)
Thanks,

Jason

_______________________________________________

freebsd-current@freebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-current

To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"

Yeah, I just realized that after looking at the memory usage of rev

1.154 (it's the same).  I could tweak kern.maxdsiz in loader.conf, but ~

1 GB is way too much memory for this program.  I know what causes the
It may not be obvious to all users and I think many will be bit by this

(i.e. POLA violation) considering maxdsiz is 512 MB on i386.  Having DM

the default would be a good idea IMHO.
Joe
--=20

Joe Marcus Clarke

FreeBSD GNOME Team      ::      gnome@FreeBSD.org

FreeNode / #freebsd-gnome

http://www.FreeBSD.org/gnome