freebsd-current mailing list

FromSubjectsort iconDate
Matthew Dillon
Re: a new way to hang 7.0
Not having a good new year, Scott? Not production ready, but certainly not vaporware. vkernel# cd /mnt vkernel# echo "abcd" > charlie vkernel# sync vkernel# hammer now 0x477eae02 vkernel# rm charlie vkernel# sync vkernel# cat charlie@@0x477eae02 abcd vkernel# ls vkernel# cd @@0x477eae02 vkernel# ls charlie vkernel# cd /mnt vkernel# echo "defgh" > charlie vkernel# sync vkernel# hammer now 0x477eae89...
Jan 4, 6:44 pm 2008
Milan Bartos
Ati driver xorg 1.4, works badly
Hi, i have problem with ati mobility radeon 9600 graphic card. With vesa driver works OK, but with ati or radeon driver, 3d acceletarion does not work (with Xorg 6.9 worked). I am now using Xorg xorg-server-1.4_3,1. xvinfo: X-Video Extension version 2.2 screen #0 Adaptor #0: "ATI Radeon Video Overlay" number of ports: 1 port base: 73 operations supported: PutImage supported visuals: depth 24, visualID 0x23 depth 24, visualID 0x24 depth 24, visualID 0x2...
Jan 4, 1:16 pm 2008
Philipp Ost
Re: Ati driver xorg 1.4, works badly
I don't have any problems with an ATi Radeon 9250 using xf86-video-ati-6.7.196 and xorg-server-1.4_3,1. This setup worked fine until X.org 7.3 was imported, it was broken until several weeks ago. I don't remember what fixed it though... This is on 7.0-PRERELEASE (i386). HTH, Philipp _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@...
Jan 4, 5:16 pm 2008
Vadim Goncharov
sbrk(2), OOM-killer and malloc() overcommit
Hi, There were talks in 'sbrk(2) broken' thread about memory hogs which can easily eat all available VM despite of resources. That'smust be fixed or it will make a bad reputation for FreeBSD as a server platform. But there are related "bug", in that of malloc overcommit, which can be demonstrated by this short program (if no resource limits are present, you'll see): #include <unistd.h> /* http://alter.tomsk.ru/bugs/head/ */ /* (C) Vladimir */ /* alter at alter tom ru */ int main...
Jan 4, 11:08 am 2008
Peter Jeremy
Re: sbrk(2), OOM-killer and malloc() overcommit
Overcommit has been bikeshedded extensively in the past. I suggest you study the archives first. --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour.
Jan 4, 3:28 pm 2008
Vadim Goncharov
Re: sbrk(2), OOM-killer and malloc() overcommit
So why we are losing users due to this "feature", as other OSes provide tunables? Can I find somewhere summary of that discussions in archives? -- WBR, Vadim Goncharov _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 4:26 pm 2008
Tim Kientzle
Re: sbrk(2), OOM-killer and malloc() overcommit
malloc overcommit is not a bug; it's an important feature for many applications, for the same reasons that sparse files are an important feature. (Many applications can optimize performance by using an addressable region much larger than the actual data they need to store.) If you really need a 4G block of memory, mmap() it to a file on disk. Tim Kientzle _______________________________________________ freebsd-current@freebsd.org mailing list [ message continues ]
" title="http://lists.freebsd.org/mailman/listinfo/freebs...">http://lists.freebsd.org/mailman/listinfo/freebs...
Jan 4, 1:36 pm 2008
Vadim Goncharov
Re: sbrk(2), OOM-killer and malloc() overcommit
That's not a solution for end-user. Why just not have space reserving, as other operating systems can guarantee to their applications that stable availability of memory? -- WBR, Vadim Goncharov _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 2:32 pm 2008
Unga
7.0-PRERELEASE installworld fails
Hi all I'm making a kernel upgrade on 7.0-BETA4 with the today's sources downloaded by cvsup. make installworld fails with following error: -------------------------------------------------------------- cd /usr/src; make -f Makefile.inc1 install ===> share/info (install) ===> lib (install) ===> lib/csu/i386-elf (install) cc -O2 -fno-strict-aliasing -pipe -I/usr/src/lib/csu/i386-elf/../common -I/usr /src/lib/csu/i386-elf/../../libc/include -Wsystem-headers -Wall -Wno-format-y2k -W...
Jan 4, 7:58 am 2008
Ivan Voras
When will ZFS become stable?
Hi, As far as I know about the details of implementation and what would it take to fix the problems, is it safe to assume ZFS will never become stable during 7.x lifetime?
Jan 4, 7:42 am 2008
Brooks Davis
Re: When will ZFS become stable?
I suppose that depends what you mean by stable. It seems stable enough for a number of applications today. It's clearly not widely tested since we haven't shipped a release based on it. It's possible some of the issues of memory requirements won't be fixable in 7.x, but I don't think that's a given. -- Brooks
Jan 4, 12:33 pm 2008
Ivan Voras
Re: When will ZFS become stable?
My yardstick is currently "when a month goes by without anyone This number is not so large. It seems to be easily crashed by rsync, for example (speaking from my own experience, and also some of my I listened to some of Pawel's talks and devsummit brainstormings and I get the feeling *none* of the problems can be fixed in 7.x, especially on i386. I'm just asking for more official confirmation. This is not a trivial question, since it involves deploying systems to be maintained some years into...
Jan 4, 1:58 pm 2008
Brooks Davis
Re: When will ZFS become stable?
I saw those crashes early one, but that's 90% of what the mirror server I'm running does and I'm not seeing them any more. I won't argue My understanding is that ZFS will never be a great choice on any 32-bit architecture without major changes Sun probably isn't interested in making. I think many of the problems people are reporting stem from I don't think anyone is naive enough to say everything will be perfect by any given date. Reality doesn't work that way. People looking to deploy ZF...
Jan 4, 2:12 pm 2008
Xin LI
Strange kernel trap 12 with vm_page_splay() on FreeBSD/i386 ...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Recently I have encountered a strange kernel trap 12, which always end up at vm_page_splay() like this, the fault va and IP vary from time to time: Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 fault virtual address = 0xfff9a4b5 fault code = supervisor read, page not present instruction pointer = 0x20:0xc073eec7 stack pointer = 0x28:0xe66fa94c frame pointer = 0x28:0xe66fa9a4 code segment = base 0x0...
Jan 3, 9:55 pm 2008
Graham Todd
Re: a new way to hang 7.0
Wow, pretty neat. If HAMMER becomes the default filesystem and snapshots are low cost and easy are they any plans to leverage these features? :-) If I understand some of the opensolaris work correctly they are starting to make use of zfs snapshots for upgrades/updates and packaging systems. _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@...
Jan 4, 3:02 pm 2008
Scott Long
Re: a new way to hang 7.0
There are better lists on which to discuss DragonflyBSD vaporware. Scott _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 3:54 pm 2008
Benjamin Close
Re: panic about half the time with WPA+WPI during startup
Hi Hans, Can you try the p4 version of the driver and see if you still have the same problems. Details about how to get it are at: http://www.clearchain.com/wiki/wpi/ Cheers, Benjamin _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 3, 8:21 pm 2008
Andrey Chernov
Re: sbrk(2) broken
Malloc() itself knows about memory amount _really_ in use by a program and could check it don't go beyond the limits, but for this it needs run-time check via getrlimit() call for each malloc() call (a program can use setrlimit() by itself). Traking direct mmap()s and sbrk()s outside of malloc() is also needed. -- http://ache.pp.ru/ _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscr...
Jan 4, 8:21 am 2008
Poul-Henning Kamp
Re: sbrk(2) broken
No, the VM system has a much better idea about this. You need to think about this the right way: There is address space allocated to the process (via sbrk/mmap) A subset of this, is address space allocated by the program (via malloc) ...and then there is memory actually in use, which is an entirely different thing, of which we currently only have some kind of clue in the VM system. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 F...
Jan 4, 8:57 am 2008
Andrey Chernov
Re: sbrk(2) broken
Then, we need sysctl to fetch that "memory actually in use" from the kernel and compare that with getrlimit() which allows malloc() to return 0 when needed. -- http://ache.pp.ru/ _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 9:12 am 2008
David Taylor
Re: sbrk(2) broken
That won't help much -- malloc could have allocated some address space that hasn't (yet) been touched by the process. Just returning 0 when the amount of memory "in use" hits a limit wouldn't stop the process from then touching all the memory it has previously been allocated and exceeding the limit. -- David Taylor _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to...
Jan 4, 9:25 am 2008
Andrey Chernov
Re: sbrk(2) broken
In that case the process is subject to be killed by system, if exceeds its limits. But... this is not malloc() problem at all, malloc() designed to detect overflow situation, not prevent it. The malloc() problem is not returning 0. -- http://ache.pp.ru/ _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 10:22 am 2008
Poul-Henning Kamp
Re: sbrk(2) broken
[Empty message]
Jan 4, 9:28 am 2008
Robert Watson
Re: sbrk(2) broken
The issue here was that there were a number of reports that out-of-control= =20 applications were toasting systems that weren't getting toasted under 6.x. = I=20 experienced this on my web server, but the ports build cluster has been=20 running into it for months. The symptom is that a single application exhau= sts=20 swap, causing all sorts of things to break (tm), killing of other large=20 processes, etc. To be clear, in the new world order, instead of getting NU= LL=20 back from malloc(3...
Jan 3, 8:26 pm 2008
Igor Mozolevsky
Re: sbrk(2) broken
Huh??? Again, huh??? _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 6:41 am 2008
Robert Watson
Re: sbrk(2) broken
FreeBSD allows memory overcommit, both overcommit of physical memory resulting in paging, and overcommit of swap space. For the last few years, resource limits on the data segment size, previously observed by malloc(), have prevented processes from mallocing enough memory individually to exhaust swap on 32-bit systems. This is arguably a bug, because you actually want a single process to be able to allocate enough memory to fill its address space, but because the data segment size is used to...
Jan 4, 7:19 am 2008
Dag-Erling Smørgrav
Re: sbrk(2) broken
For the same reason as it has for the last 20 years or so: memory overcommit, which means that malloc() allocates address space, not memory. Actual memory is allocated on-demand when the address space is used (read from or written to). If there is no RAM left and none can be freed by swapping out, the process gets killed. The process that gets killed is not necessarily the memory hog, it is merely the process that is unlucky enough to touch a new page at the wrong moment, i.e. when all RAM and s...
Jan 4, 6:55 am 2008
Igor Mozolevsky
Re: sbrk(2) broken
Broadcasting SIGDANGER would be a much better option; followed by SIGTERM to the memory hogger (to allow for graceful termination) and only then SIGKILL. I can imagine a few (legitimate) scenarios when a That would be really cool and even better if it allocated it in a contiguous chunk. Igor :-) _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current...
Jan 4, 7:18 am 2008
Dag-Erling Smørgrav
Re: sbrk(2) broken
We don't currently have SIGDANGER, but the signal code was rewritten years ago to allow more than 32 signals precisely for the purpose of implementing an AIX-like SIGDANGER. This wasn't done, however, and eventually SIGTHR was the first new signal to take advantage of the No. First of all, you're thinking of lseek(), not fseek() Second, an lseek() beyond the end of a file will not actually extend the file. Third, ftruncate() (which *will* extend a file if it is shorter than the requested length...
Jan 4, 8:45 am 2008
Poul-Henning Kamp
Re: sbrk(2) broken
In message <86myrlahee.fsf@ds4.des.no>, =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= wr SIGDANGER is not what we need. What we need is an intelligent mechanism to tell applications what the overall situation is, so that jemalloc and aware applications can tune their usage pattern to the availability of physical and virtual memory. Instead of the binary "SIGDANGER" indication we need a more gradual state, at the very least three stats: "plenty", "getting a bit tight" and "crunchtime". Having ...
Jan 4, 8:53 am 2008
Igor Mozolevsky
Re: sbrk(2) broken
This makes memory management in the userland hideously and unnecessarily complicated. It's simpler to have SIGDANGER (meaning, free all you can) -> SIGTERM (terminate gracefully) -> SIGKILL (too late, I'm killing you anyway); and maybe a MIB in sysctl like ...vm.overcommit_action ='soft' being SIGDANGER->SIGTERM->SIGKILL and = 'hard' being SIGKILL, so the sysadmin at least has a choice Igor _______________________________________________ freebsd-current@freebsd.org mailing list htt...
Jan 4, 9:03 am 2008
Dag-Erling Smørgrav
Re: sbrk(2) broken
You don't seem to understand what Poul-Henning was trying to point out, which is that broadcasting SIGDANGER can make a bad situation much, much worse by waking up and paging in every single process in the system, including processes that are blocked and wouldn't otherwise run for several minutes, hours or even days (getty, inetd, sshd, mountd, even nfsd / nfsiod in some cases can sleep for days at a time waiting for I/O) DES -- Dag-Erling Smørgrav - des@des.no ______________________________...
Jan 4, 9:12 am 2008
Kostik Belousov
Re: sbrk(2) broken
By making the default action for SIGDANGER to be SIG_IGN, this problem would be mostly solved. Only processes that actually care about SIGDANGER and installing the handler for it would require some non-trivial and resource-hungry operation.
Jan 4, 9:48 am 2008
Robert Watson
Re: sbrk(2) broken
Another aspect of the problem is that applications have come to depend in= =20 malloc(3) returning NULL when memory is getting tight, and while we have ne= ver=20 done exactly that, we have historically had malloc(3) return NULL when we g= et=20 close to the process data segment size. Robert N M Watson Computer Laboratory University of Cambridge
Jan 4, 9:24 am 2008
Kris Kennaway
Re: sbrk(2) broken
Do everyone a favour and research the topic in the archives, please. Another thread on the subject will just waste everyone's time. Kris _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 7:31 am 2008
Robert Watson
Re: sbrk(2) broken
That will create a sparse file without file system blocks to back it, and is effectively also over-commit. When the file system runs out of room, you will get SIGSEGV when the vnode pager discovers it can't write a page to disk. If you zero-fill it, the blocks are pre-allocated. In a more ideal world, we might support an ioctl or system call to pre-allocate but not hook up the blocks until they were written to, in order to avoid writing lots of zeros to disk, but we don't live in that ideal...
Jan 4, 7:22 am 2008
Igor Mozolevsky
Re: sbrk(2) broken
Surely you should not be allowed to overcommit on fseek() followed by write(,,1); zeroing out gigs of hdd space seems rather silly... Igor _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 7:30 am 2008
Robert Watson
Re: sbrk(2) broken
Sparse files are a feature. It just becomes inconvenient at that point because you discover the lack of space asynchronously from a useful user process event. When memory pressure gets high, the vnode pager decides it's time to push a dirty page to disk, and then discovers that there are no free blocks on the file system to write to. As I mentioned in my e-mail, it would be nice if our file system supported a way to reserve blocks for files without hooking them up to the file's visiible add...
Jan 4, 7:38 am 2008
Dag-Erling Smørgrav
Re: sbrk(2) broken
Even for files which are intended to be filled up immediately, telling the file system ahead of time how much data will be written would allow it to make much better layout decisions. DES -- Dag-Erling Smørgrav - des@des.no _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 8:48 am 2008
Dag-Erling Smørgrav
Re: sbrk(2) broken
Not a good solution on its own. You need a per-process limit as well, otherwise a malloc() bomb will still cause other processes to fail Thank you :) DES -- Dag-Erling Smørgrav - des@des.no _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Jan 4, 5:32 am 2008
Robert Watson
Re: sbrk(2) broken
ly. That was what I had in mind, the above should read RLIMIT_SWAP. Robert N M Watson Computer Laboratory University of Cambridge
Jan 4, 7:06 am 2008
Skip Ford
Re: sbrk(2) broken
Robert Watson wrote: > On Fri, 4 Jan 2008, Dag-Erling Sm
Jan 4, 9:54 am 2008
Kostik Belousov
Re: sbrk(2) broken
Oh, I thought that I was the sole user of the patch. What problems did you encountered while testing it ? What you mean by "do 90% of swap" ?
Jan 4, 9:59 am 2008
Skip Ford
Re: sbrk(2) broken
> > > On Fri, 4 Jan 2008, Dag-Erling Sm
Jan 4, 10:11 am 2008
Kostik Belousov
Re: sbrk(2) broken
Ok. The patch really imposes two kind of limits: - the total amount of anon memory that could be allocated in the whole system (this is what I called "disabling overcommit") - per-user RLIMIT_SWAP limit, that account the allocation by the uid. This has some obvious problems with setuid(2) syscall. AFAIR, I ended up not moving the accounted numbers to the new uid. Both limits can be turned on/off independently. May be, time to revive it.
Jan 4, 10:18 am 2008
Skip Ford
Re: sbrk(2) broken
> > > > > On Fri, 4 Jan 2008, Dag-Erling Sm
Jan 4, 10:58 am 2008
Dag-Erling Smørgrav
Re: sbrk(2) broken
You don't want the default to be so high. You want a low default, with the possibility for the admin to increase the limit for a particular user in login.conf or similar without rebooting (which is currently not possible since the default datasize == maxdsiz, which can only be changed in the kernel config or loader.conf) You may also want to have a collective limit for unprivileged users, so root will still be able to log in if something goes wrong. DES -- Dag-Erling Smørgrav - des@des.no ...
Jan 4, 8:34 am 2008
Robert Watson
Re: sbrk(2) broken
This will presumably only work for console logins, as sshd (etc) will depen= d=20 on unprivileged users, but perhaps that is fine. I'm less concerned with t= he=20 details of the implementation or policy than that we simply be able to supp= ort=20 even a basic policy and have it configured by default to prevent=20 foot-shooting. Robert N M Watson Computer Laboratory University of Cambridge
Jan 4, 9:26 am 2008
Ian FREISLICH
Re: sbrk(2) broken
I'm not sure that I like that very much. At least the way that it has been explained here so correct me if I misunderstood. I have long lived processes that continuously handle very valuable data and potentially get very large (several GB). I'd like that process to be able to make a rational decision about what happens to its memory contents when an allocation fails rather than having the proverbial rug pulled out from under it. Rug pulling at any point can cost an annual salary or two. Ia...
Jan 4, 2:27 am 2008
Peter Jeremy
Re: sbrk(2) broken
[Empty message]
Jan 4, 5:51 am 2008
previous daytodaynext day
January 3, 2008January 4, 2008January 5, 2008