Hello,

Now that the release is over, I'll try to get rid of zones and migrate
allocations to objcache.

Here is a 1st patch that migrate struct proc.  I'll commit it in a
couple of days if nobody objects.

I'm also implementing a way to monitor objcache statistics.  So far,
I've hacked sth similar to
sysctl vm.zone: stats are returned in one giant string, one line per
cache.  That's simple but ugly and hard to read because there are
quite a few stats per cache.

So I'm thinking about implementing a small tool, ocstat, that extracts
stats from sysctl the way ps(1) does (thru libkvm iirc) and formats it
in various ways.  What do you think?

Cheers,
Nicolas

Index: src2/sys/kern/kern_exit.c
===================================================================
--- src2.orig/sys/kern/kern_exit.c	2008-03-31 08:05:59.139796000 +0200
+++ src2/sys/kern/kern_exit.c	2008-03-31 10:31:39.000000000 +0200
@@ -834,7 +834,7 @@ loop:
 			}

 			vm_waitproc(p);
-			zfree(proc_zone, p);
+			objcache_put(proc_cache, p);
 			nprocs--;
 			return (0);
 		}
Index: src2/sys/kern/kern_proc.c
===================================================================
--- src2.orig/sys/kern/kern_proc.c	2008-03-31 08:05:59.139918000 +0200
+++ src2/sys/kern/kern_proc.c	2008-03-31 10:31:39.000000000 +0200
@@ -84,7 +84,7 @@ u_long pgrphash;
 struct proclist allproc;
 struct proclist zombproc;
 struct spinlock allproc_spin;
-vm_zone_t proc_zone;
+struct objcache* proc_cache;
 vm_zone_t lwp_zone;
 vm_zone_t thread_zone;

@@ -131,7 +131,7 @@ procinit(void)
 	spin_init(&allproc_spin);
 	pidhashtbl = hashinit(maxproc / 4, M_PROC, &pidhash);
 	pgrphashtbl = hashinit(maxproc / 4, M_PROC, &pgrphash);
-	proc_zone = zinit("PROC", sizeof (struct proc), 0, 0, 5);
+	proc_cache = objcache_create_simple(M_PROC, sizeof(struct proc));
 	lwp_zone = zinit("LWP", sizeof (struct lwp), 0, 0, 5);
 	thread_zone = zinit("THREAD", sizeof (struct thread), 0, 0, 5);
 	uihashinit();
Index: src2/sys/sys/proc.h
====...
[ message continues ]

The object cache is not a general-purpose memory allocator.  It's a cache of
pre-initialized objects.  Are the significant portions of the proc structure
that can be restored to initialized state on object puts?  If not, it's best
to just use a general-purpose memory allocator.

								Jeffrey

Well, I haven't checked this yet.  I planned to do this in a second
step after getting rid of zones.

I thought that even without  the pre-initialization part, objcache has
the advantage
of pre-allocating set of objects (as zones) and is additionally
mp-safe (contrary to zones and, granted, as malloc()).   The latter
may be interesting as we talk of improving MP support.

Cheers,
Nicolas

>> Are there significant portions of the proc structure
  >> that can be restored to initialized state on object puts?

  > I planned to do this in a second step after getting rid of zones.

I'll all in favor of making more use of caching pre-initialized objects, so
if this is your objective, then please go ahead.  It's just that I'd convert
over to the object cache in one step.

  > I thought that even without  the pre-initialization part, objcache has
  > the advantage of pre-allocating set of objects (as zones) and is additionally
  > mp-safe (contrary to zones and, granted, as malloc()).

Well, the zone allocator is MP-safe.  It's not hard to lock up the places that
manipulate the linked-list of free items. Look for uses of the zlock field.

								Jeffrey

Well it is documented as MP-safe but it isn't actually and is deprecated:

http://leaf.dragonflybsd.org/mailarchive/kernel/2008-01/msg00045.html

Actually, the above thread is what triggered me into migrating stuff off zones.

Thanks for all your feedback.  I'll precise my current plan in my
forthcoming reply to Matt.

Nicolas

>> Well, the zone allocator is MP-safe.  It's not hard to lock up the places
  >> that manipulate the linked-list of free items. Look for uses of the zlock
  >> field.

  > Well it is documented as MP-safe but it isn't actually and is deprecated:

  > http://leaf.dragonflybsd.org/mailarchive/kernel/2008-01/msg00045.html

I think you misread that post.  The zone allocator itself is MP-safe.
(Again, look for the zlock spinlock in the zone allocator code to
see for yourself that it's MP-safe.)  The post is refering to the
subsystems that use the zone allocators as being not MP-safe.

							Jeffrey

Indeed but the first line of that post also states that none of the VM
s/system is MP safe.  As the zone implementation calls into the VM
without explicitly taking the MP lock, it implicitly assumes that its
callers do so.

That being said, I agree that zlock protects the zone data structures
though I wonder about a few cases.  For example, what happens if
zget() is called twice concurrently on the same ZONE_INTERRUPT zone?
I reckon we could end up calling twice vm_page_alloc() on the same
page.

These races could be fixed of course but AFAIU Matt would rather get
rid of zones.

Cheers,
Nicolas

Why?  What's the loss if we use the object cache to allocate struct procs=
?

cheers
  simon

The object cache sits on top of a general-purpose memory allocator such
as kmalloc or zalloc and provides pre-initialized object caching functionality.
If you're not going to use the object caching functionality, you can just
call the underlying memory allocator directly.

							Jeffrey