quoted text > I see Allen beat me to the reply with the requested netstat data
> below, but in the mean time, I'm going to do the unthinkable and build
> a custom kernel with your mods and see where the chips fall. :-)
>
> Thanks for the suggestion.
>
> Kevin
>
>
>
> On Thu, May 15, 2008 at 2:45 PM, Allen <justnix2000@gmail.com> wrote:
>> On Thu, May 15, 2008 at 2:00 PM, Darrian Hale <djh@rgnets.com> wrote:
>>> What output to you get from 'netstat -m'?
>>
>> 2867 mbufs in use:
>> 2566 mbufs allocated to data
>> 274 mbufs allocated to packet headers
>> 27 mbufs allocated to socket names and addresses
>> 1129/5450/6144 mbuf clusters in use (current/peak/max)
>> 13028 Kbytes allocated to network (22% in use)
>> 0 requests for memory denied
>> 0 requests for memory delayed
>> 0 calls to protocol drain routines
>>
>>
>>
>>> I might get yelled at for this as you mentioned people seem to hate
>>> custom kernels.
>>>
>>> But i've had good luck with the following options, I'm not sure which
>>> are still relevant, but they help.
>>>
>>> option NKMEMPAGES_MAX=81920
>>> option NKMEMPAGES=81920
>>> option MAX_KMAPENT=8192
>>>
>>> I've always received that error you described on any high load openbsd
>>> box. Even with the above changes,
>>> you will eventually get the same error as your new limits are reached.
>>>
>>> If you come up with any better solutions, please let me know, i'd be
>>> very interested to hear them.
>>>
>>> -Darrian
>>>
>>> On Thu, May 15, 2008 at 10:29 AM, Kevin <spytag@gmail.com> wrote:
>>>> All,
>>>>
>>>> I'm getting quite a lot of these errors in /var/log/messages and can't
>>>> seem to find an appropriate fix in the archives:
>>>>
>>>> May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries
>>>> May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries
>>>> May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries
>>>> May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries
>>>> May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries
>>>>
>>>> N.B. This machine serves mirror content for various F/OSS projects in
>>>> addition to standard www content, so it quite often has >350 users
>>>> concurrently connected downloading mirrored content (in addition to
>>>> visitors who're actually visiting the site).
>>>>
>>>>
>>>> These messages correspond almost exactly with two things:
>>>>
>>>> 1.) the sites having quite a few visitors
>>>> 2.) the sites becoming unavailable. In most cases, it fixes itself
>>>> when the freeloaders (errr downloaders ;-) complete their file
>>>> transfers.
>>>>
>>>>
>>>> Possibly worth noting:
>>>>
>>>> 1.) We've had to crank various settings in Apache to keep serving
>>>> traffic, as the stock settings were too low: we were reaching the max
>>>> daemons for Apache so new visitors were just out-of-luck.
>>>> 2.) When the system begins to knuckle under load, I'm taking a
>>>> snapshot of various bits like the following.
>>>>
>>>> Here's one example:
>>>> load averages: 0.45, 0.47, 0.40 07:40:00
>>>> 247 processes: 245 idle, 2 on processor
>>>> CPU0 states: 7.2% user, 0.0% nice, 2.6% system, 2.2% interrupt, 88.0% idle
>>>> CPU1 states: 3.6% user, 0.0% nice, 0.3% system, 1.9% interrupt, 94.3% idle
>>>> Memory: Real: 339M/737M act/tot Free: 1272M Swap: 0K/518M used/tot
>>>>
>>>>
>>>> From the archives this seems to be something for which a fix *used* to
>>>> be cranking up the following:
>>>>
>>>> maxusers 64
>>>> option BUFCACHEPERCENT=25
>>>> option MULTIPROCESSOR
>>>> option MAX_KMAPENT=4000
>>>>
>>>>
>>>> This hardly seems a real fix though--especially given everyone's
>>>> hatred of knobs, custom kernels, and such though I'm certainly open to
>>>> it if we can continue to keep the sites--and the mirrors--up.
>>>>
>>>> I think I've mentioned everything noteworthy though cluestick
>>>> applications are welcome.
>>>>
>>>> Thanks,
>>>> Kevin
>>>>
>>>>
>>>>
>>>>
>>>> Here's the dmesg for any interested parties:
>>>>
>>>>
>>>> OpenBSD 4.3 (GENERIC.MP) #2: Fri Apr 11 09:00:02 PDT 2008
>>>> kevin@svr02:/usr/src/sys/arch/i386/compile/GENERIC.MP
>>>> cpu0: Intel(R) Pentium(R) III CPU family 1266MHz ("GenuineIntel"
>>>> 686-class) 1.27 GHz
>>>> cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE
>>>> real mem = 2146992128 (2047MB)
>>>> avail mem = 2067959808 (1972MB)
>>>> mainbus0 at root
>>>> bios0 at mainbus0: AT/286+ BIOS, date 01/25/02, BIOS32 rev. 0 @
>>>> 0xffe90, SMBIOS rev. 2.3 @ 0xfae20 (49 entries)
>>>> bios0: vendor Dell Computer Corporation version "A06" date 01/25/2002
>>>> bios0: Dell Computer Corporation PowerEdge 2550
>>>> acpi0 at bios0: rev 0
>>>> acpi0: tables DSDT FACP APIC SPCR
>>>> acpi0: wakeup devices PCI1(S5) PCI2(S5) PCI0(S5)
>>>> acpitimer0 at acpi0: 3579545 Hz, 32 bits
>>>> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
>>>> cpu0 at mainbus0: apid 1 (boot processor)
>>>> cpu0: apic clock running at 132MHz
>>>> cpu1 at mainbus0: apid 0 (application processor)
>>>> cpu1: Intel(R) Pentium(R) III CPU family 1266MHz ("GenuineIntel"
>>>> 686-class) 1.27 GHz
>>>> cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE
>>>> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 16 pins
>>>> ioapic0: misconfigured as apic 0, remapped to apid 2
>>>> ioapic1 at mainbus0: apid 3 pa 0xfec01000, version 11, 16 pins
>>>> ioapic1: misconfigured as apic 0, remapped to apid 3
>>>> acpiprt0 at acpi0: bus 0 (PCI1)
>>>> acpiprt1 at acpi0: bus 1 (PCI2)
>>>> acpiprt2 at acpi0: bus 2 (PCI0)
>>>> acpiprt3 at acpi0: bus 3 (I960)
>>>> acpicpu0 at acpi0
>>>> acpicpu1 at acpi0
>>>> bios0: ROM list: 0xc0000/0x8000 0xcc000/0x8000 0xec000/0x4000!
>>>> esm0 at mainbus0
>>>> esm0: PowerEdge 2550 Embedded Server Management 5.50
>>>> esm0: Primary System Backplane 1.30
>>>> pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
>>>> pchb0 at pci0 dev 0 function 0 "ServerWorks CNB20HE Host" rev 0x23
>>>> pchb1 at pci0 dev 0 function 1 "ServerWorks CNB20HE Host" rev 0x01
>>>> pci1 at pchb1 bus 2
>>>> ppb0 at pci1 dev 2 function 0 "Intel i960 RM PCI-PCI" rev 0x02
>>>> pci2 at ppb0 bus 3
>>>> ahc0 at pci2 dev 4 function 0 "Adaptec AIC-7899 U160" rev 0x01: apic 3
>>>> int 15 (irq 11)
>>>> scsibus0 at ahc0: 16 targets
>>>> ahc1 at pci2 dev 4 function 1 "Adaptec AIC-7899 U160" rev 0x01: apic 3
>>>> int 14 (irq 10)
>>>> scsibus1 at ahc1: 16 targets
>>>> fxp0 at pci1 dev 4 function 0 "Intel 8255x" rev 0x08, i82559: apic 3
>>>> int 0 (irq 5), address 00:06:5b:3b:61:27
>>>> inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4
>>>> pchb2 at pci0 dev 0 function 2 "ServerWorks CNB20HE Host" rev 0x01
>>>> pchb3 at pci0 dev 0 function 3 "ServerWorks CNB20HE Host" rev 0x01
>>>> pci3 at pchb3 bus 1
>>>> bge0 at pci3 dev 8 function 0 "Broadcom BCM5700" rev 0x12, BCM5700 B2
>>>> (0x7102): apic 3 int 1 (irq 10), address 00:06:5b:3b:61:28
>>>> brgphy0 at bge0 phy 1: BCM5401 10/100/1000baseT PHY, rev. 3
>>>> gdt0 at pci0 dev 4 function 0 "Vortex GDT6x18RD" rev 0x00: apic 3 int
>>>> 6 (irq 11) dpmem c8000 2-bus 2 cache devices
>>>> gdt0: ver 11a, cache on, strategy 2, writeback on, blksz 32
>>>> gdt0: raw feat 1 cache feat 101
>>>> scsibus2 at gdt0: 35 targets
>>>> sd0 at scsibus2 targ 0 lun 0: <ICP, Host drive #00, > SCSI2 0/direct fixed
>>>> sd0: 35000MB, 4462 cyl, 255 head, 63 sec, 512 bytes/sec, 71682030 sec total
>>>> sd1 at scsibus2 targ 1 lun 0: <ICP, Host drive #01, > SCSI2 0/direct fixed
>>>> sd1: 140200MB, 17873 cyl, 255 head, 63 sec, 512 bytes/sec, 287129745 sec total
>>>> scsibus3 at gdt0: 16 targets
>>>> scsibus4 at gdt0: 16 targets
>>>> vga1 at pci0 dev 14 function 0 "ATI Rage XL" rev 0x27
>>>> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
>>>> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
>>>> piixpm0 at pci0 dev 15 function 0 "ServerWorks OSB4" rev 0x50: SMBus disabled
>>>> pciide0 at pci0 dev 15 function 1 "ServerWorks OSB4 IDE" rev 0x00: DMA
>>>> atapiscsi0 at pciide0 channel 0 drive 0
>>>> scsibus5 at atapiscsi0: 2 targets
>>>> cd0 at scsibus5 targ 0 lun 0: <TEAC, CD-224E, 3.7D> SCSI0 5/cdrom removable
>>>> cd0(pciide0:0:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 2
>>>> isa0 at mainbus0
>>>> isadma0 at isa0
>>>> pckbc0 at isa0 port 0x60/5
>>>> pckbd0 at pckbc0 (kbd slot)
>>>> pckbc0: using irq 1 for kbd slot
>>>> wskbd0 at pckbd0: console keyboard, using wsdisplay0
>>>> pcppi0 at isa0 port 0x61
>>>> midi0 at pcppi0: <PC speaker>
>>>> spkr0 at pcppi0
>>>> lpt0 at isa0 port 0x378/4 irq 7
>>>> npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
>>>> pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
>>>> pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
>>>> fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
>>>> fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
>>>> mtrr: Pentium Pro MTRR support
>>>> softraid0 at root
>>>> root on sd0a swap on sd0b dump on sd0b
