On Thu, Jul 24, 2008 at 11:07:36AM +0100, Mindaugas Rasiukevicius wrote:
quoted text
> Jason Thorpe <thorpej@shagadelic.org> wrote:
> >=20
> > On Jul 12, 2008, at 10:07 AM, Gary Thorpe wrote:
> >=20
> > > The advantages of SA are supposed to be on *I/O* bound workloads =20
> > > because it can reduce overhead due to kernel context switches by =20
> > > doing userland switches where appropriate (according to previous =20
> > > benchmarks, research etc.), but I could be wron on this.
> >=20
> > No.  I/O bound switching in SA requires a kernel context switch =20
> > (because the thread blocks in the kernel).
>=20
> Not only context switch, but it also needs a new LWP (kernel thread) when
> blocking. In a case of new pthread (userland) or cold "LWP-cache" (actual=
ly,
quoted text
> it is pool) it means: creation of new LWP + context switch. Also, at some
> point LWP should be destroyed or put back to the "LWP-cache", what means
> more overhead, or well.. wasting of memory.
>=20
> Imagine a case of 1000 (new) pthreads which block - that would mean:
> 1000 * (LWP creation + SA context switch) operations. Plus, LWPs for VPs.=
..

That would be the exact same LWP usage as a 1:1 threading model would=20
give. The SA process spends the time creating the LWPs between blocking=20
events while the 1:1 process created all of the same LWPs at initial=20
thread creation time.

Yes, the SA process also has an extra LWP sitting around per VP, but that=
=20
is a constant so it really only should count against process startup.

One other thing to consider is how long different context switches take.=20
The two important ones are intra-process-same-space switches (inter-LWP in=
=20
the kernel and inter-thread in SA userland) and user-kernel switches. When=
=20
I was starting the Wasabi iSCSI target, I asked around before we used (SA)=
=20
pthreads to implement this. I asked a number of NetBSD threading folks=20
about this.

The answer I was given was that user-kernel switches are NOTABLY more=20
expensive. Like 10x. Their numbers, not mine. So while SA is adding extra=
=20
steps, they are steps that aren't the most expensive thing around.

What I don't understand, though, is why we're discussing this issue like=20
this. I don't see what the NetBSD kernel loses by having both 1:1 AND SA=20
threading support. While the SA code is a fresh port, it is a fresh port=20
of the NetBSD 4 code. So it actually is something we're familiar with as a=
=20
project. People on this list have shown that SA does better on some work=20
loads, and other people have shown (quite spectacularly) that 1:1 performs=
=20
stunningly.

Yes, we had a nasty discussion when 1:1 was brought into current. But=20
looking back, I think most of the nastiness was due to the fact that it=20
was presented as an either-or proposition. We now have an entirely=20
differnt case. Re-adding SA does NOT mean losing 1:1.

Take care,

Bill