Work continues to progress well but I've hit a few snags:
    * My current recovery model is just not working for cross-cluster

      transactions.
    * I can't fix the write performance issue due to the way the cluster

      localization and spike code works.
    * Handling nearly full filesystems is proving to be a problem due to

      the inability to estimate space requirements which in turn are

      related to the cluster localization the current model uses.
    * Efficient real-time mirroring is just not easy to do w/ the current

      cluster-localized model.
    Everything else now works, and works well, including and most especially

    the historical access features.  I've already fallen in love with being

    able to create a softlink to a '@@0x<timestamp>' string to access

    snapshots.
    I've come to the conclusion that I am going to have to make a fairly

    radical change to the on-disk structures to solve these problems.  On

    the plus side, these changes will greatly simplify the filesystem topology

    and greatly reduce its complexity.  On the negative side, recovering

    space will not be instantanious and will basically require data to

    be copied from one part of the filesystem to another.  My take is that

    since large filesystems (our target) do not usually fill up very quickly,

    and indeed it can take hours even if you are writing 60MB/sec to the disk,

    It is well worth taking a hit on free space recovery if it accomplishes

    all of the filesystem's other goals.
    This may seem a bit radical as solutions go, but the more I look at it

    the more I like it because it really does neatly tie up every single

    remaining issue.
    * Get rid of super-clusters, clusters, and typed buffers.  Get rid of them

      entirely.  Just have an array of buffers after the volume header.
    * Get rid of the A-list's (the recursive bitmap radix tree allocation

      model).  Poof, gone.  All of them, gone.  Have no random ...
[ message continues ]

he,
I would propose to delay the 2.0 release a bit so that you have enough

time to hack on that issue?!?  IMO its better to delay the release for

some weeks and have a bit more stable HAMMER instead.
Regards
	Matthias

There's always the option of releasing a 1.12 version now (it's not like

there haven't been enough changes to justify a new release). The 2.0 release

is likely to get a lot of downloads, so I think shipping it with a pre-alpha

hammer is a waste of an opportunity to attract more people. Not to mention

that it's hard to put a time bound on this kind of development. So, let's

just admit that and ship 2.0 as soon as Matt declares it ready for beta

testing, regardless of what time of year it is. A real beta-state hammer

justifies a 2.0 release on its own IMHO. Also, this will let Matt work on

hammer without any tight deadlines.
Aggelos

On Wednesday 06 February 2008, Matthew Dillon wrote:
Just curious: what was the write performance issue? I mean, what was the use

case and how bad was the performance? I don't think it was described in any

detail in your past HAMMER update mails, but please point me to the mail if 
As long as I'm asking HAMMER-related questions... This may be obvious from the

code but I haven't looked at it yet. Will it be possible to disable the

historical access feature and still have the possibility to take a snapshot

(of the filesystem or a directory tree) at some point in time?
Thanks,

Aggelos

ta.
How will this affect parallel IO (reads, but especially writes)? Would=20

having such a global structure serialize it? (I'm assuming, possibly=20

wrongly, that having trees per-cluster allowed you to lock individual=20

clusters).

This sounds quite like LFS now.  LFS however split the volume in smaller

blocks which could be "used", "empty" or "open", IIRC.  Their background

cleaner then could push remaining data from used blocks to a currently

open one, marking the block "empty" after that, allowing the FS to write

to the blocks again.
They always had problems with the cleaner, I think.
cheers

   simon
--

Serve - BSD     +++  RENT this banner advert  +++    ASCII Ribbon   /"\

Work - Mac      +++  space for low €€€ NOW!1  +++      Campaign     \ /

Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \

Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \

How about "Generational Garbarge Collection"? Assuming that there are

some files that will never be deleted this could give slighly better

performance.
Keep a "copy count" (a copy occurs if the cleaner has to copy data from

the left end to the right end of the FIFO). If that increases over, say

3, copy it into the old generation FIFO.
One problem of course is how to dimension each generation, and how many

to use.
I think that's basically how LFS works.
Regards,
   Michael