Re: [PATCH 2/2] improve ext3 fsync batching

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Andrew Morton
Date: Tuesday, August 19, 2008 - 10:56 am

On Tue, 19 Aug 2008 07:01:11 -0400 Ric Wheeler <rwheeler@redhat.com> wrote:


Obviously, tuning that delay down to the minimum necessary is a good
thing.  But doing it based on commit-time seems indirect at best.  What
happens on a slower disk when commit times are in the tens of
milliseconds?  When someone runs a concurrent `dd if=/dev/zero of=foo'
when commit times go up to seconds?

Perhaps a better scheme would be to tune it based on how many other
processes are joining that transaction.  If it's "zero" then decrease
the timeout.  But one would need to work out how to increase it, which
perhaps could be done by detecting the case where process A runs an
fsync when a commit is currently in progress, and that commit was
caused by process B's fsync.

But before doing all that I would recommend/ask that the following be
investigated:

- How effective is the present code?

  - What happens when it is simply removed?

  - Add instrumentation (a counter and a printk) to work out how
    many other tasks are joining this task's transaction.

    - If the answer is "zero" or "small", work out why.

  - See if we can increase its effectiveness.

Because it could be that the code broke.  There might be issues with
higher-level locks which are preventing the batching.  For example, if
all the files which the test app is syncing are in the same directory,
perhaps all the tasks are piling up on that directory's i_mutex?
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH 1/2] add hrtimer_sleep_ns helper function, Josef Bacik, (Wed Aug 6, 12:08 pm)
[PATCH 2/2] improve ext3 fsync batching, Josef Bacik, (Wed Aug 6, 12:15 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Josef Bacik, (Wed Aug 6, 12:23 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Andrew Morton, (Mon Aug 18, 9:31 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Andreas Dilger, (Mon Aug 18, 10:44 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Ric Wheeler, (Tue Aug 19, 4:01 am)
Re: [PATCH 2/2] improve ext3 fsync batching, Andrew Morton, (Tue Aug 19, 10:56 am)
Re: [PATCH 2/2] improve ext3 fsync batching, Ric Wheeler, (Tue Aug 19, 11:08 am)
Re: [PATCH 2/2] improve ext3 fsync batching, Ric Wheeler, (Tue Aug 19, 11:43 am)
Re: [PATCH 1/2] add hrtimer_sleep_ns helper function, Matthew Wilcox, (Tue Aug 19, 12:15 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Josef Bacik, (Tue Aug 19, 12:18 pm)
Re: [PATCH 1/2] add hrtimer_sleep_ns helper function, Josef Bacik, (Tue Aug 19, 12:22 pm)
Re: [PATCH 1/2] add hrtimer_sleep_ns helper function, Matthew Wilcox, (Tue Aug 19, 12:36 pm)
Re: [PATCH 1/2] add hrtimer_sleep_ns helper function, Josef Bacik, (Tue Aug 19, 12:39 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Andrew Morton, (Tue Aug 19, 1:29 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Andrew Morton, (Tue Aug 19, 1:34 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Ric Wheeler, (Tue Aug 19, 1:55 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Andrew Morton, (Tue Aug 19, 2:18 pm)
Re: [PATCH 2/2] improve ext3 fsync batching, Ric Wheeler, (Tue Aug 19, 2:29 pm)