Hi Andrew, Jens, James,

The following patches are the revised version to use lld's private data
instead of backing_dev_info.state for exporting lld busy state.

Please review and give me your opinion.
If all of you have no problem, I'll re-send the patches separately
to each maintainer, Jens and James.

The patches are created on the following commit of for-2.6.28
in linux-2.6-block, but the scsi part can be also applied to
scsi-post-merge-2.6.
---------------------------------------------------------------
commit c5062758edbe59286d7d8e718a59e4d4347ef541
Author: Jens Axboe <jens.axboe@oracle.com>
Date:   Wed Sep 24 13:05:10 2008 +0200

    libata: set queue SSD flag for SSD devices
---------------------------------------------------------------


Previous post and discussions about the interface:
    http://lkml.org/lkml/2008/9/19/138


Summary of the patches:
    1/2: block: add lld busy state exporting interface
    2/2: scsi: export busy state via q->lld_busy_fn()

Thanks,
Kiyoshi Ueda
--

[ message continues ]

This patch adds an new interface, blk_lld_busy(), to check lld's
busy state from the block layer.
blk_lld_busy() calls down into low-level drivers for the checking
if the drivers set q->lld_busy_fn() using blk_queue_lld_busy().

This resolves a performance problem on request stacking devices below.

Some drivers like scsi mid layer stop dispatching request when
they detect busy state on its low-level device like host/target/device.
It allows other requests to stay in the I/O scheduler's queue
for a chance of merging.

Request stacking drivers like request-based dm should follow
the same logic.
However, there is no generic interface for the stacked device
to check if the underlying device(s) are busy.
If the request stacking driver dispatches and submits requests to
the busy underlying device, the requests will stay in
the underlying device's queue without a chance of merging.
This causes performance problem on burst I/O load.

With this patch, busy state of the underlying device is exported
via q->lld_busy_fn().  So the request stacking driver can check it
and stop dispatching requests if busy.

The underlying device driver must return the busy state appropriately:
    1: when the device driver can't process requests immediately.
    0: when the device driver can process requests immediately,
       including abnormal situations where the device driver needs
       to kill all requests.


Signed-off-by: Kiyoshi Ueda <k-ueda@ct.jp.nec.com>
Signed-off-by: Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Jens Axboe <jens.axboe@oracle.com>
---
 block/blk-core.c       |   25 +++++++++++++++++++++++++
 block/blk-settings.c   |    6 ++++++
 include/linux/blkdev.h |    4 ++++
 3 files changed, 35 insertions(+)

Index: linux-2.6-block/block/blk-settings.c
===================================================================
--- linux-2.6-block.orig/block/blk-settings.c
+++ linux-2.6-block/block/blk-settings.c
@@ -89,6 +89,12 @@ void ...
[ message continues ]

This patch implements q->lld_busy_fn() for scsi mid layer to export
its busy state.

Please note that it checks the cached information (sdev->lld_busy)
for efficiency, instead of checking the actual state of
sdev/starget/shost everytime.

So the care must be taken not to leave sdev->lld_busy = 1 for
the following cases:
    - when there is no request in the sdev queue
    - when scsi can't dispatch I/Os anymore and needs to kill I/Os
Otherwise, request stacking drivers may not submit any request,
and then, nobody sets back lld_busy = 0 and that causes deadlock.

OTOH, it has no major problem in setting sdev->lld_busy = 0 even when
the starget/shost is actually busy, because newly submitted request
will soon turn it to 1 in scsi_request_fn().


Signed-off-by: Kiyoshi Ueda <k-ueda@ct.jp.nec.com>
Signed-off-by: Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
---
 drivers/scsi/scsi.c        |    4 ++--
 drivers/scsi/scsi_lib.c    |   20 +++++++++++++++++++-
 include/scsi/scsi_device.h |   13 +++++++++++++
 3 files changed, 34 insertions(+), 3 deletions(-)

Index: linux-2.6-block/drivers/scsi/scsi_lib.c
===================================================================
--- linux-2.6-block.orig/drivers/scsi/scsi_lib.c
+++ linux-2.6-block/drivers/scsi/scsi_lib.c
@@ -470,6 +470,8 @@ void scsi_device_unbusy(struct scsi_devi
 	spin_unlock(shost->host_lock);
 	spin_lock(sdev->request_queue->queue_lock);
 	sdev->device_busy--;
+	if (sdev->device_busy < sdev->queue_depth && !sdev->device_blocked)
+		sdev->lld_busy = 0;
 	spin_unlock_irqrestore(sdev->request_queue->queue_lock, flags);
 }
 
@@ -1461,6 +1463,13 @@ static void scsi_softirq_done(struct req
 	}
 }
 
+static int scsi_lld_busy(struct request_queue *q)
+{
+	struct scsi_device *sdev = q->queuedata;
+
+	return sdev ? sdev->lld_busy : 0;
+}
+
 /*
  * Function:    scsi_request_fn()
  *
@@ -1503,9 +1512,14 @@ static void scsi_request_fn(struct reque
 		 * ...
[ message continues ]