Re: [00/41] Large Blocksize Support V7 (adds memmap support)

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Mel Gorman
Date: Thursday, September 13, 2007 - 2:40 am

On (12/09/07 16:17), Christoph Lameter didst pronounce:

One of Nick's points is that to have a 100% reliable solution, that is
what is required. We already have a layering between the VM and the FS
but my understanding is that fsblock replaces rather than adds to it.

Surely, we'll be able to detect the situation where the memory is really
contiguous as a fast path and have a slower path where fragmentation was
a problem.


This is going in circles.

His point is that we also cannot prove it is 100% correct in all
situations. Without taking additional (expensive) steps, there will be a
workload that fragments physical memory. He doesn't know what it is and neither
do we, but that does not mean that someone else will find it. He also has a
point about the slow degredation of fragmentation that is woefully difficult
to reproduce. We've had this provability of correctness problem before.

His initial problem was not with the patches as such but the fact that they
seemed to be presented as a 1st class feature that we fully support and
is a potential solution for some VM and IO Scalability problems.  This is
not the case, we have to treat it as a 2nd class feature until we *know* no
situation exists where it breaks down. These patches on their own would have
to run for months if not a year or so before we could be really sure about it.

The only implementation question about these patches that hasn't been addressed
is the mmap() support. What's wrong with it in it's current form. Can it be
fixed or if it's fundamentally screwed etc. That has fallen by the
wayside.


Unless callers always use an iterator for blocks that is optimised in the
physically linear case to be a simple array offset and when not physically
linear it either walks chains (complex) or uses vmap (must deal with TLB
flushes amoung other things). If it optimistically uses physically contiguous
memory, we may find a way to use only one page struct as well.


Or gee whiz, I don't know. Start with your patches as a strictly 2nd class
citizen and build fsblock in while trying to keep use of physically contiguous
memory where possible and it makes sense.


I am *very* wary of using reserve pools for anything other than
emergency situations. If nothing else pools == wasted memory + a sizing
problem. But hey, it is one option.

Are we going to agree on some sort of plan or are we just going to
handwave ourselves to death?

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab
-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 10, 11:03 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 5:05 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Tue Sep 11, 9:02 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 9:47 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Maxim Levitsky, (Tue Sep 11, 11:25 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 12:20 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:01 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:03 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:07 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:11 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:13 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:41 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:42 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:35 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:41 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:48 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:52 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 4:26 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 5:00 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 5:04 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Wed Sep 12, 1:20 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Martin J. Bligh, (Wed Sep 12, 7:29 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Wed Sep 12, 4:06 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Wed Sep 12, 4:17 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Mel Gorman, (Thu Sep 13, 2:40 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Thu Sep 13, 7:38 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Fri Sep 14, 9:10 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Fri Sep 14, 10:52 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Fri Sep 14, 11:08 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Fri Sep 14, 11:15 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Fri Sep 14, 5:31 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Fri Sep 14, 5:33 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 15, 5:14 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sat Sep 15, 8:51 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 15, 1:14 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sat Sep 15, 3:30 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 6:54 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sun Sep 16, 8:08 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Sun Sep 16, 11:15 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Sun Sep 16, 11:44 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sun Sep 16, 11:50 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sun Sep 16, 2:31 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 2:58 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:06 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:38 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:48 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:51 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:56 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:00 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:03 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:05 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:10 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:21 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Tue Sep 18, 11:30 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 18, 12:18 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 18, 12:31 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Tue Sep 18, 12:44 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 18, 1:36 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 18, 1:42 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Wed Sep 19, 7:04 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Thu Sep 20, 7:54 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Thu Sep 20, 11:07 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Thu Sep 20, 11:11 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 10:50 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 11:22 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 11:49 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 11:56 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Mon Sep 24, 8:39 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 24, 2:13 pm)