Andrew,

This patchset mainly polishes the writeback queuing policies.
The main goals are:

(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
    - don't leave them out; no congestion_wait() in between them
(3) avoid busy iowait for blocked inodes
    - retry them in the next go of s_io(maybe at the next wakeup of pdflush)

The role of the queues:

s_dirty:   park for dirtied_when expiration
s_io:      park for io submission
s_more_io: for big dirty inodes, they will be retried in this run of pdflush
           (it ensures fairness between small/large files)
s_more_io_wait: for blocked inodes, they will be picked up in next run of s_io


This patchset is in better shape, but still not ready for merge.
It begins with:

	[PATCH 01/13] writeback: revert 2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b      
	[PATCH 02/13] writeback: clear PAGECACHE_TAG_DIRTY for truncated page in block_write_full_page()

Introduces more_io/more_io_wait based policies:

	[PATCH 03/13] writeback: introduce writeback_control.more_io                  
	[PATCH 04/13] writeback: introduce super_block.s_more_io_wait                 
	[PATCH 05/13] writeback: merge duplicate code into writeback_some_pages()     
	[PATCH 06/13] writeback: defer writeback on not-all-pages-written             
	[PATCH 07/13] writeback: defer writeback on locked inode                      
	[PATCH 08/13] writeback: defer writeback on locked buffers                    
	[PATCH 09/13] writeback: requeue_io() on redirtied inode                      

And finishes with some code cleanups:

	[PATCH 10/13] writeback: introduce queue_dirty()                              
	[PATCH 11/13] writeback: queue_dirty() on memory-backed bdi                   
	[PATCH 12/13] writeback: remove redirty_tail()                                
	[PATCH 13/13] writeback: cleanup __sync_single_inode()                        

Diffstat:

 fs/buffer.c         |    2 
 ...
[ message continues ]

Fengguang do you have any specific tests for any of these cases? As I
have posted earlier I am putting together a writeback test suite for
test.kernel.org and if you have one (even if it's an ugly shell
script) that would save me some time.

Also if you want any of mine let me know. :-)

-

[ message continues ]

No, I just run tests with cp/dd etc.  I analyze the code and debug
traces a lot, and know that it works in the situations I can imagine.

OK, thank you.

Fengguang

-

[ message continues ]

Quick question to make sure I get this. Each queue is sorted as such:

s_dirty - sorted by the dirtied_when field
s_io - sorted by  no explicit key but by the order we want to process
in sync_sb_inodes
s_more_io - held for later they are sorted in the same manner as s_io

Is that it?

mrubin
-

[ message continues ]

Yes, exactly. s_io and s_more_io can be considered as one list broken
up into two - to provide the cursor for sequential iteration.
And s_more_io_wait is simply a container for blocked inodes.

-

[ message continues ]