Re: endless sync on bdi_sched_wait()? 2.6.33.1

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Dave Chinner
Date: Sunday, April 11, 2010 - 5:47 pm

On Thu, Apr 08, 2010 at 11:28:50AM +0200, Jan Kara wrote:
....

Jan, just another data point that i haven't had a chance to look
into yet - I noticed that 2.6.34-rc1 writeback patterns have changed
on XFS from looking at blocktrace.

The bdi-flush background write threadi almost never completes - it
blocks in get_request() and it is doing 1-2 page IOs. If I do a
large dd write, the writeback thread starts with 512k IOs for a
short while, then suddenly degrades to 1-2 page IOs that get merged
in the elevator to 512k IOs.

My theory is that the inode is getting dirtied by the concurrent
write() and the inode is never moving back to the dirty list and
having it's dirtied_when time reset  - it's being moved to the
b_more_io list in writeback_single_inode(), wbc->more_io is being
set, and then we re-enter writeback_inodes_wb() which splices the
b_more_io list back onto the b_io list and we try to write it out
again.

Because I have so many dirty pages in memory, nr_pages is quite high
and this pattern continues for some time until it is exhausted, at
which time throttling triggers background sync to run again and the
1-2 page IO pattern continues.

And for sync(), nr_pages is set to LONG_MAX, so regardless of how
many pages were dirty, if we keep dirtying pages it will stay in
this loop until LONG_MAX pages are written....

Anyway, that's my theory - if we had trace points in the writeback
code, I could confirm/deny this straight away. First thing I need to
do, though, is to forward port the original writeback tracng code
Jens posted a while back....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Jan Kara, (Thu Apr 8, 2:28 am)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Denys Fedorysychenko, (Thu Apr 8, 3:12 am)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Dave Chinner, (Sun Apr 11, 5:47 pm)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Dave Chinner, (Sun Apr 18, 6:37 pm)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Dave Chinner, (Mon Apr 19, 12:23 am)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Jan Kara, (Tue Apr 20, 5:33 pm)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Dave Chinner, (Tue Apr 20, 6:54 pm)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Jan Kara, (Wed Apr 21, 6:27 am)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Dave Chinner, (Wed Apr 21, 5:06 pm)
Re: endless sync on bdi_sched_wait()? 2.6.33.1, Jan Kara, (Thu Apr 22, 5:48 am)