Just to stick my two cents in here:
The definition of what is meant by "large" filesystems has to change
with the advances in disk drive technology. In the not too distant
past, a "large" single filesystem was 100 GB. There are now consumer
grade disks on the market with 1 TB available in a single unit. I don't
know about you guys, but that scares the crap out of me, in terms of
dealing with that much space on a desktop machine. Efficiently dealing
with transferring that much data on a desktop (never mind server) means
re-thinking the limitations of the I/O subsystems. What was once the
realm of the data center is now the realm of the living room. Large
data sets are becoming more commonplace (HD Movies, audio files, etc)
with each passing day, and there is no end in sight in the progression.
In addition, with the release of specs recently about larger sector
sizes for disk drives (2048 bytes, or larger), this is going to become a
pressing need for the general case, not just the extremely large
servers, or HPC machines and clusters. Already there is no efficient
way to back up that much space, in a reasonable time, except to have
another disk of a similar or larger size to back up to. Anything we can
do to make disk I/O *Faster* is a win.
I recognize that there is a huge issue in dealing with sub block size
files. The trade off of small files VS large blocks is now a non
trivial problem. Once disk sector sizes increase, the problems will
have to be dealt with in a more intelligent manner, possibly dividing
sectors into smaller logical blocks for small files? Maybe filesystems
that can understand multiple block sizes?
Well, we do live in interesting times; we just have to make the most of
it.
Dan Weigert
-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of Eric W.
Biederman
Sent: Monday, May 07, 2007 2:56 AM
To: David Chinner
Cc: Andrew Morton; clameter@sgi.com; linux-kernel@vger.kernel.org; Mel
Gorman; William Lee Irwin III; Jens Axboe; Badari Pulavarty; Maxim
Levitsky
Subject: Re: [00/17] Large Blocksize Support V3
David Chinner <dgc@sgi.com> writes:
Ok.
for
Yes, and but it isn't a generic implementation in mm/filemap.c,
it is a compatibility layer. It lives with the current deficiencies
instead of removes them.
Small scatter lists?
Always?
Ugh. I just realized looking at the xfs code that it doesn't
work in the presence of high memory, at least not with 4K pages.
But that is how mm/filemap.c works. The calls into the filesystem
can be per multi-page group as they are current per page. The point
is that the existing in kernel abstraction is already larger then a
page for doing the work.
Yes. You are talking about only fixing the kernel for your giant
64K block filesystems that are only interesting on peta-byte arrays.
I am pointing out that the other fixes that have been discussed.
Optimistic contiguous page allocation and a larger linux scatter
gather list. Are interesting on much smaller filesystem and machine
sizes where small files are still interesting. Making them generally
better improvements for linux.
If you only improve the giant peta-byte raid cases 99% of linux users
simply don't care, and so the code isn't very interesting.
Eric
-
