Hi,
I'm currently trying to address port-i386/1635. Since the same issue
might be present on other archs, and this needs change to generic
code in any case, I'm cross-posting both port-i386@ and tech-kern@.

There is a problem presented in the PR that NetBSD/i386 can actually
run maximum 8192 processes at the same time due to how many GDT
slots are allocated. If kern.maxproc limit is set high enough to
allow more than 8192 processes, kernel would panic in gdt_get_slot()
once there is enough running processes. Easy way to trigger this
panic would be to set kern.maxproc=10000, change current process
limits high enough and run forkbomb like:

quoted text
> cat me.c
int
main() {
	while(1) {
		if (fork() == 0)
			execl("./me", "me", NULL);
	}
}
quoted text
>

My solution to this is adding optional arch-specific hook
to kern_sysctl.c, which would not allow setting maxproc to
higher than arch-specific maximum. Patch is below.
It does not deal with maxproc being bumped via NPROC kernel
option; I assume that people using NPROC option know what
they are doing. Perhaps a #ifdef DEBUG check in init_main() might
be appropriate for this.

Comments?
Is the ((MAXGDTSIZ - NGDT)/2) really needed in USER_LDT case for i386?

Thanks,

Jaromir

Index: kern/kern_sysctl.c
===================================================================
RCS file: /cvsroot/syssrc/sys/kern/kern_sysctl.c,v
retrieving revision 1.121
diff -u -p -r1.121 kern_sysctl.c
--- kern/kern_sysctl.c	2002/12/12 20:54:58	1.121
+++ kern/kern_sysctl.c	2002/12/12 21:17:45
@@ -385,6 +385,10 @@ kern_sysctl(int *name, u_int namelen, vo
 			if (nmaxproc < 0 || nmaxproc >= PID_MAX - PID_SKIP)
 				return (EINVAL);
 
+#ifdef __HAVE_CPU_MAXPROC
+			if (nmaxproc > cpu_maxproc())
+				return (EINVAL);
+#endif
 			maxproc = nmaxproc;
 		}
 
Index: arch/i386/include/cpu.h
===================================================================
RCS file: /cvsroot/syssrc/sys/arch/i386/include/cpu.h,v
retrieving revision 1.92
diff -u -p -r1.92 cpu.h
--- arch/i386/include/cpu.h	2002/12/06 14:47:09	1.92
+++ arch/i386/include/cpu.h	2002/12/12 21:17:45
@@ -355,6 +355,7 @@ extern int i386_has_sse2;
 
 /* machdep.c */
 void	dumpconf __P((void));
+int	cpu_maxproc __P((void));
 void	cpu_reset __P((void));
 void	i386_init_pcb_tss_ldt __P((struct cpu_info *));
 void	i386_proc0_tss_ldt_init __P((void));
Index: arch/i386/i386/machdep.c
===================================================================
RCS file: /cvsroot/syssrc/sys/arch/i386/i386/machdep.c,v
retrieving revision 1.506
diff -u -p -r1.506 machdep.c
--- arch/i386/i386/machdep.c	2002/12/07 15:36:20	1.506
+++ arch/i386/i386/machdep.c	2002/12/12 21:17:45
@@ -3784,3 +3784,16 @@ idt_vec_free(vec)
 	idt_allocmap[vec] = 0;
 	simple_unlock(&idt_lock);
 }
+
+/*
+ * Number of processes is limited by number of available GDT slots.
+ */
+int
+cpu_maxproc(void)
+{
+#ifdef USER_LDT
+	return ((MAXGDTSIZ - NGDT) / 2);
+#else
+	return (MAXGDTSIZ - NGDT);
+#endif
+}
Index: arch/i386/include/types.h
===================================================================
RCS file: /cvsroot/syssrc/sys/arch/i386/include/types.h,v
retrieving revision 1.39
diff -u -p -r1.39 types.h
--- arch/i386/include/types.h	2002/10/01 12:57:09	1.39
+++ arch/i386/include/types.h	2002/12/12 21:48:36
@@ -71,5 +71,6 @@ typedef int		register_t;
 #define	__HAVE_GENERIC_SOFT_INTERRUPTS
 #define __HAVE_RAS
 #define	__HAVE_MD_RUNQUEUE
+#define	__HAVE_CPU_MAXPROC
 
 #endif	/* _MACHTYPES_H_ */
-- 
Jaromir Dolecek <jdolecek@NetBSD.org>            http://www.NetBSD.org/
-=- We should be mindful of the potential goal, but as the tantric    -=-
-=- Buddhist masters say, ``You may notice during meditation that you -=-
-=- sometimes levitate or glow.   Do not let this distract you.''     -=-