On Mon, Jun 23, 2008 at 12:53 PM, Christopher S. Aker
<caker@theshore.net> wrote:
quoted text
> adam radford wrote:
>>
>> On Mon, Jun 23, 2008 at 9:15 AM, Christopher S. Aker <caker@theshore.net>
>> wrote:
>>
>>> It's happened on a number of different hosts, all of the same hardware
>>> and
>>> software configuration (Xen 3.2 64bit, 32bit pae dom0, 32bit pae domUs.
>>>  LVM
>>> backend with 3ware hardware RAID-1).
>
>>
>>
>> A driver patch for older kernels including XenServer-4.1 is available
>> here:
>>
>> http://www.3ware.com/KB/article.aspx?id=15243&cNode=6I1C6S
>
> Thanks, but unfortunately, from that link:
>
> "3ware 9000 series controllers are not affected by this issue."
>
> ... which is what we're using.  This problem only appeared after rebooting
> these machines into Xen.  Some of affected boxes even ran 2.6.18 (non Xen)
> for awhile without any problems.
>
> -Chris
>

I've seen cases where Xen DMA code (pci-dma-xen.c) sends I/O requests
un-necessarily through the SWIOTLB code.

Xen dma: avoid unnecessarily SWIOTLB bounce buffering:

http://lists.xensource.com/archives/html/xen-changelog/2008-04/msg00008.html

so you may need a similar bidirectional fix for the 3w-9xxx driver.

Can you try this patch (in-line below and also attached):

-Adam

diff -Naur linux-2.6.18-xen-3.2.0/drivers/scsi/3w-9xxx.c
linux-2.6.18-xen-3.2.0.new/drivers/scsi/3w-9xxx.c
--- linux-2.6.18-xen-3.2.0/drivers/scsi/3w-9xxx.c	2008-02-12
02:44:33.000000000 -0800
+++ linux-2.6.18-xen-3.2.0.new/drivers/scsi/3w-9xxx.c	2008-06-23
15:42:02.000000000 -0700
@@ -1388,7 +1388,7 @@
 	if (cmd->use_sg == 0)
 		goto out;

-	use_sg = pci_map_sg(pdev, cmd->request_buffer, cmd->use_sg,
DMA_BIDIRECTIONAL);
+	use_sg = pci_map_sg(pdev, cmd->request_buffer, cmd->use_sg,
cmd->sc_data_direction);

 	if (use_sg == 0) {
 		TW_PRINTK(tw_dev->host, TW_DRIVER, 0x1c, "Failed to map scatter
gather list");
@@ -1415,7 +1415,7 @@
 		goto out;
 	}

-	mapping = pci_map_single(pdev, cmd->request_buffer,
cmd->request_bufflen, DMA_BIDIRECTIONAL);
+	mapping = pci_map_single(pdev, cmd->request_buffer,
cmd->request_bufflen, cmd->sc_data_direction);

 	if (mapping == 0) {
 		TW_PRINTK(tw_dev->host, TW_DRIVER, 0x1d, "Failed to map page");
@@ -1423,7 +1423,7 @@
 	}

 	cmd->SCp.phase = TW_PHASE_SINGLE;
-	cmd->SCp.have_data_in = mapping;
+	cmd->SCp.dma_handle = mapping;
 	retval = mapping;
 out:
 	return retval;
@@ -2000,10 +2000,10 @@

 	switch(cmd->SCp.phase) {
 	case TW_PHASE_SINGLE:
-		pci_unmap_single(pdev, cmd->SCp.have_data_in, cmd->request_bufflen,
DMA_BIDIRECTIONAL);
+		pci_unmap_single(pdev, cmd->SCp.dma_handle, cmd->request_bufflen,
cmd->sc_data_direction);
 		break;
 	case TW_PHASE_SGLIST:
-		pci_unmap_sg(pdev, cmd->request_buffer, cmd->use_sg, DMA_BIDIRECTIONAL);
+		pci_unmap_sg(pdev, cmd->request_buffer, cmd->use_sg, cmd->sc_data_direction);
 		break;
 	}
 } /* End twa_unmap_scsi_data() */