Ok, here's the final design document that I am now implementing.

    Again, I expect most or all of these features to be ready and the

    filesystem to be beta-quality by the December release.
			       Hammer Filesystem
(I) General Storage Abstraction
    HAMMER uses a basic 16K filesystem buffer for all I/O.  Buffers are

    collected into clusters, cluster are collected into volumes, and a

    single HAMMER filesystem may span multiple volumes.
    HAMMER maintains a small hinted radix tree for block management in

    each layer.  A small radix tree in the volume header manages cluster

    allocations within a volume, one in the cluster header manages buffer

    allocations within a cluster, and most buffers (pure data buffers

    excepted) will embed a small tree to manage item allocations within

    the buffer.
    Volumes are typically specified as disk partitions, with one volume

    designated as the root volume containing the root cluster.  The root

    cluster does not need to be contained in volume 0 nor does it have to

    be located at any particular offset.
    Data can be migrated on a cluster-by-cluster or volume-by-volume basis

    and any given volume may be expanded or contracted while the filesystem

    is live.   Whole volumes can be added and (with appropriate data

    migration) removed.
    HAMMER's storage management limits it to 32768 volumes, 32768 clusters

    per volume, and 32768 16K filesystem buffers per cluster.   A volume

    is thus limited to 16TB and a HAMMER filesystem as a whole is limited

    to 524288TB.  HAMMER's on-disk structures are designed to allow future

    expansion through expansion of these limits.  In particular, the volume

    id is intended to be expanded to a full 32 bits in the future and using

    a larger buffer size will also greatly increase the cluster and volume

    size limitations by increasing the number of elements the buffer-

    restricted radix trees can manage.
    HAMMER breaks all of its ...
[ message continues ]

Hi,
I hope this question has not been implicitly answered before, but how

does Hammer handle quotas? Filesystems like XFS and ZFS maintain quota

information internally so that a quotacheck after a system crash does

not take ages. It seems to me that Hammer could manage quotas as a

part of its cluster allocation strategy. Is this the case?
TIA,

RIggs

Wow, this seems pretty good.
What about data corruption issues ?
Have you thought about implementing some sort of checksumming mechanism ?
We cannot assume hardware to be absolutely reliable. There may be some

silent corruption going on the disk or network layers, etc...
More on this in this article:

http://kerneltrap.org/Linux/Data_Errors_During_Drive_Communication
--

Francois Tigeot

Quoting from Matt's announcement:
"    All information in a HAMMER filesystem is CRCd to detect corruption."
'All'
So the question - if there is one - is 'how good' that check is.
Otherwise, not the fs' job.
It *must* presume a 'generally reliable' environment beyond a certain point.
Error prevention, detection, (possible) correction, and friends more properly

should exist in the storage hardware, I/O, and link layers.
As they do. Or do not.
.. just as the article you cited points out.... hardware and driver selection

issues, or even suboptimal silicon.
Bill

According to Matt's design document:
     "All information in a HAMMER filesystem is CRCd to detect

      corruption."
Regards,
   Michael

Any specific reason not to go with a B+-Tree or B#-Tree which have shown

to have advantageous effects?
Also, what, if any, will be the locking policy for multiple I/O threads

accessing a single HAMMER filesystem? Shared/exclusive mutexes on the

cluster level?
(Whomever invented early mornings does not deserve brownie points),

--

         Thomas E. Spanjaard

         tgen@netphreax.net

Interesting. What exactly are those database files used for? Is a

database file attached to each file to store ACLs, for example? Or can

it be used like btree(3)? Do they have their own namespace in the

filesystem?
Wow! I a am really looking forward to try out HAMMER!!!
Regards,
   Michael

Because HAMMER uses a B-Tree (maybe a B+Tree the more I look at it)..

     in anycase, because HAMMER uses a B-Tree all lookups are basically

     key searches, even when looking up an offset in a file.  Since B-Tree

     elements specify records which can reference variable-length data,

     there really is very little difference between a database record

     indexed with a key and regular file data indexed with an offset.
     Records are typed so any given filesystem object can contain multiple

     key spaces.  One space will hold ACLs, one will be for regular file

     offsets, and there's nothing preventing us from having a key space

     directly accessible by userland.
     A HAMMER-aware database would be able to store its records using the

     key space directly.  It opens up some intriguing possibilities.
					    -Matt

					    Matthew Dillon

					    <dillon@backplane.com>

.. including implementing a ZFS-like DB/fs crossbreed *ATOP* HAMMER.
Or a Venti workalike (surpassed already on feature-set, but not storage

efficiency AFAICS).
IF one really wanted to do either badly enough.  And had a need.
Too complicated for my taste, but PostgreSQL data store might be another matter

entirely.
Bill

*snip*
Matt,
Awesome!
Tells me: "ZFS, bend over, grab your ankles and kiss your an(atomy) 'Goodbye'"
 From the amount of work that has HAD to go into this, it also tells me you are:
A) probably single, or soon will be and
B) don't sleep much anyway!
;-)
Looking forward to a 'test drive'...
Bill Hacker