Back in March I posted some MySQL benchmarks after we switched to a 1:1
threading model in -current *. I've spent a lot of time tuning the pthread
library so I thought I'd post a followup. The original benchmark that I used
(supersmack) now performs much better on -current that it did a few months
ago, so I picked something else this time: MySQL sysbench.

Most of the sysbench runs that I've seen to date have sysbench running on
the same machine as the database. That's a good test but with the exception
of small installations and out-of-band activity, production setups rarely
look like that. So I ran sysbench itself on a seperate dual core system.

Here are the results, comparing NetBSD 3 with NetBSD-current:

        http://www.netbsd.org/~ad/sysbench/netbsd.png

And NetBSD-current compared to other systems:

        http://www.netbsd.org/~ad/sysbench/netbsd-and-others.png

Note this is stock NetBSD-current with FreeBSD's malloc() (jemalloc) in
libc. I'll be merging that some time soon.

With the vmlocking CVS branch and Mindaugas' new scheduler NetBSD peaks
around 500 TPS. There is a very gradual fall off in the number of TPS
achieved as the number of connections begins to ramp up. I suspect that
could be due to a weakness somewhere in the network stack, so I'm hopeful
that a bit of time spent profiling with large numbers of connections could
yield good results.

Thanks,
Andrew

* http://mail-index.netbsd.org/tech-kern/2007/03/02/0005.html

This is all very impressive. Congratulations -- clearly your work has
been paying off.

It would be interesting if you could include x86 Solaris in future
benchmarks.

Perry

On Fri, 28 Sep 2007, Perry E. Metzger wrote:

http://www.netbsd.org/~ober/
Some basic tests on a 4x P-III 500mhz Xeon with 2MB L2.
I created one million rows and ran sysbench in oltp mode in both default 
ReadWrite, and Readonly.

Did not make use of ZFS or any other optimizations.
Will be testing the recommendations for Mysql on Solaris.



Jaime Fournier
ober@netbsd.org

Nice! If you fancy doing more tests, here are some files that might be of
use:

http://www.netbsd.org/~ad/sysbench/my.cnf
http://www.netbsd.org/~ad/sysbench/jemalloc-libc.tar.gz
http://www.netbsd.org/~ad/sysbench/start.sh
http://www.netbsd.org/~ad/sysbench/netbsd.bz2

start.sh ups the limits and sets LD_LIBRARY_PATH so that MySQL is using the
libc with jemalloc. netbsd.bz2 is a kernel from the 'vmlocking' branch. It
will panic if you unmount a file system, so make sure you do a few syncs
before shutting the machine down if you do run it.. If you experiment with
driving the machine with sysbench remotely, the host needs to be reasonably
powerful (say, a 3GHz Pentium 4) or it will not be able to keep the DB fully
loaded.

Thanks,
Andrew

Something that Jeff Roberson mentioned to me (but that I have not tried to
reproduce) is that once you do a read/write run, the performance of the DB
is permanently changed. So it may make sense to reinitialize the DB if doing
another read-only run.

Andrew

Interesting indeed!  Thanks for that feedback!

Darren

It would be a brighter place if you stopped pushing people around and
weren't such a deadweight. How about you benchmark it?

Andrew

Andrew, there's a lot of merit in including Solaris10/OpenSolaris 
benchmarks in the
result as that's where a lot of big databases run today...numbers from 
[Open]Solaris
are much more relevant in this type of measure than OpenBSD - they're 
only relevant
if we want to see want NetBSD 2.0 or older was like.

In addition to stating the CPUs, can you please include the disk 
configuration info.?

Cheers,
Darren

I tried to install S10U2 but my install media are scratched. A few years
back when I installed Solaris x86, the I2O drivers didn't work with DPT /
Adaptec RAID controllers (and I hear that they are slated for removal in
'Nevada'). The machine also has a Compaq RAID controller but from what I
remember swapping the disks onto that means wiping the contents. I will
give it a go when I have some free time.

I also tried Dragonfly but despite hours trying I couldn't get an SMP

There is some detail in my e-mail to Warner. I'm using single, removable
disks at 80MB/s but it doesn't really matter since the tests I ran were
read-only. A read-write test would be limited by the disk so eventually, all
the systems would likely produce similar graphs. On the other hand, if
someone has an external FC array that they'd like to donate/lend, I can do a
read-write test. :-)

Andrew

Darren, thats maybe a reason for a benchmark and not how Perry did ask
in his usual way (I can understand Andrews response).

Andrew, thank you very much for all the work you are doing here.

Regards,
Bernd

On Sat, 29 Sep 2007 09:04:24 +0200

I saw nothing wrong in how Perry asked.  Look at the text above -- he
simply remarked that it would be interesting.  It also isn't reasonable
to say "you do it" when the goal is to get comparable numbers -- you
want the same test setup, which Andrew has.

Indeed, on this and other matters.  My thanks, too. 



		--Steve Bellovin, http://www.cs.columbia.edu/~smb

Thanks for the positive attitude.

.pm