Re: HAMMER update 11-June-2008

Previous thread: Re: GSoC 2008 dma enhancements by Matthew Dillon on Wednesday, June 11, 2008 - 8:08 pm. (1 message)

Next thread: Re: HAMMER update 11-June-2008 by Matthew Dillon on Friday, June 13, 2008 - 1:10 pm. (1 message)
To: <kernel@...>
Date: Wednesday, June 11, 2008 - 8:27 pm

After another round of performance tuning HAMMER all my benchmarks
show HAMMER within 10% of UFS's performance, and it beats the shit
out of UFS in certain tests such as file creation and random write
performance. Read performance is good but drops more then UFS under
heavy write loads (but write performance is much better at the same
time).

I am making progress with blogbench. It turns out that HAMMER isn't
quite as horrible as it first appeared. What was happening was simply
that the test under UFS never got past blog #200, and thus never wrote
enough data to blow out the system caches. The blogbench test builds
up an ever-increasing sized dataset as it progresses.

HAMMER has superior random write performance and because of that it
easily got into the blog #350-400 range in the default test, or about
double the size of the data-set. Plus it was writing more
during the first half of the test so that of course depressed the
read performance a bit relative to UFS.

When I increase the number of iterations sufficiently for both UFS and
HAMMER to blow out the system caches, the results wind up being very
different.

blogbench --iterations=100 -d /mnt/bench2

Now when UFS gets past blog #300 and blows out the system caches, UFS's
write performance goes completely to hell but it is able to maintain
good read performance:

Nb blogs R articles W articles R pictures W pictures R comments W comments
322 72232 67 55654 88 45740 194
323 83711 81 64882 81 53844 204
325 57380 62 43314 62 36603 196
...
345 17494 40 12866 50 12226 137
347 21895 42 16655 41 13002 128
347 22803 68 17517 12 14247 122
348 16976 52 13397 29 12113 119
...

To: <kernel@...>
Date: Thursday, June 12, 2008 - 10:48 pm

On Thu, Jun 12, 2008 at 10:27 AM, Matthew Dillon

This is really great to hear. Making a new filesystem from scratch is
hard enough already, and you've done it quite successfully on your
own, within the limitations of the current DragonFly kernel. I dearly
hope this gets even higher mindshare than DragonFly itself, because it
obviously has the potential to be highly useful on other systems as
well.

By the way, have you looked into POHMELFS? I don't know if it's really
the same kind of design as HAMMER, but it does solve some similar
problems and is also a pretty strong performer.

--
Dmitri Nikulin

Centre for Synchrotron Science
Monash University
Victoria 3800, Australia

To: <kernel@...>
Date: Friday, June 13, 2008 - 8:59 am

POHMELFS (http://tservice.net.ru/~s0mbre/old/?section=projects&item=pohmelfs)
is a network filesystem, thus not very similar to HAMMER (and the performance
measurements vary a lot as Evgeniy Polyakov adds/removes features). A very
interesting project anyway.

You were probably thinking of btrfs
(http://btrfs.wiki.kernel.org/index.php/Main_Page) on top of which Zach Brown
and friends are building crfs (http://oss.oracle.com/projects/crfs/). These
two seem to be moving in the direction Matt wants to take. Unfortunately I
don't have the time to follow their development (only so many hours in a
day).

HTH,
Aggelos

To: <kernel@...>
Date: Saturday, June 14, 2008 - 11:01 pm

On Fri, Jun 13, 2008 at 10:59 PM, Aggelos Economopoulos

Yes of course, I was only referring to Matt's network part of the
filesystem. I forgot all about btrfs since I haven't heard much news
of it lately, but POHMELFS is in my RSS feeds every day.

--
Dmitri Nikulin

Centre for Synchrotron Science
Monash University
Victoria 3800, Australia

Previous thread: Re: GSoC 2008 dma enhancements by Matthew Dillon on Wednesday, June 11, 2008 - 8:08 pm. (1 message)

Next thread: Re: HAMMER update 11-June-2008 by Matthew Dillon on Friday, June 13, 2008 - 1:10 pm. (1 message)