Use TLB batching for MADV_FREE.  Adds another 10-15% extra performance
to the MySQL sysbench results on my quad core system.

Signed-off-by: Rik van Riel <riel@redhat.com>
---

Nick Piggin wrote:

quoted text
>> 3) because of this, we can treat any such accesses as
>>    happening simultaneously with the MADV_FREE and
>>    as illegal, aka undefined behaviour territory and
>>    we do not need to worry about them
> 
> Yes, but I'm wondering if it is legal in all architectures.

It's similar to trying to access memory during an munmap.

You may be able to for a short time, but it'll come back to
haunt you.

quoted text
>> 4) because we flush the tlb before releasing the page
>>    table lock, other CPUs cannot remove this page from
>>    the address space - they will block on the page
>>    table lock before looking at this pte
> 
> We don't when the ptl is split.

Even then we do.  Each invocation of zap_pte_range() only touches
one page table page, and it flushes the TLB before releasing the
page table lock.

quoted text
> What the tlb flush used to be able to assume is that the page
> has been removed from the pagetables when they are put in the
> tlb flush batch.

All the tlb flush code seems to assume is that the tlb entries
should be invalidated.

quoted text
> I'm not saying there is any bugs, but just suggesting there
> might be.

Jakub found a potential bug, in that I did not use an atomic
operation to clear the page table entries.  I've attached a
new patch which simply uses ptep_test_and_clear_dirty/young
to get rid of the dirty and accessed bits.

It uses the same atomic accesses we use elsewhere in the VM
and the code is a line shorter than before.

Andrew, please use this one.

-- 
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.