[PATCH 0/2]Add Variable Page Size and IA64 Support in Intel IOMMU

Previous thread: [PATCH] Rationalise email addresses by Alan Cox on Wednesday, October 1, 2008 - 9:53 am. (1 message)

Next thread: [PATCH 1/2]Add Variable Page Size and IA64 Support in Intel IOMMU: Generic Part by Fenghua Yu on Wednesday, October 1, 2008 - 9:57 am. (9 messages)
From: Fenghua Yu
Date: Wednesday, October 1, 2008 - 9:56 am

The following two patches (Intel IOMMU generic patch and ia64 specific patch) together enable Intel IOMMU on IA64 platform. They are applied cleanly on the latest linux-next tree which contains Intel IOMMU interrupt remapping and VT-d driver KVM support code.

If applying only the generic patch on x86-64 kernel, x86-64 IOMMU is working fine, i.e. the generic patch doesn't have compilation and functionality impact on x86-64 IOMMU. If applying only the ia64 specific patch on ia64 kernel, ia64 is working as non-IOMMU/swiotlb case. If applying both of the patches and set CONFIG_DMAR=y, ia64 IOMMU is working.

Thanks.

-Fenghua
--

From: Bjorn Helgaas
Date: Friday, October 3, 2008 - 8:50 am

General comment (not ia64-related): it looks like you detect the
IOMMU using the static DMAR table.  Does the IOMMU also appear in
PCI config space or the ACPI namespace?  I expect it would, because
the fact that the DMAR table is static precludes any sort of hot-plug
to add or remove IOMMUs.

If it *is* in config space or the namespace, it would be good to
exercise that discovery path to help shake out firmware bugs.  For
example, arch/ia64/hp/common/sba_iommu.c uses acpi_bus_register_driver()
to discover HP IOMMUs.  I wouldn't hold the sba_iommu.c discovery code
(most of which I take the blame for) up as a shining example of how to
do things, but it shows the point.

Bjorn
--

From: Yu, Fenghua
Date: Monday, October 6, 2008 - 12:44 pm

The latest VT-d spec v1.1 does not support hot plug. If the feature is available in newer spec, we will support it.

Thanks.

-Fenghua
--

From: Fenghua Yu
Date: Monday, October 6, 2008 - 5:01 pm

The patche set (Intel IOMMU generic patch and ia64 specific patch) together enable Intel IOMMU on IA64 platform. They are applied cleanly on the latest linux-next tree which contains Intel IOMMU interrupt remapping and VT-d driver KVM support code.
 
If applying only the generic patch on x86-64 kernel, x86-64 IOMMU is working fine, i.e. the generic patch doesn't have compilation and functionality impact on x86-64 IOMMU. If applying only the ia64 specific patch on ia64 kernel, ia64 is working as non-IOMMU/swiotlb case. If applying both of the patches and set CONFIG_DMAR=y, ia64 IOMMU is working.
 
Thanks.
 
-Fenghua
--

From: Fenghua Yu
Date: Wednesday, February 11, 2009 - 11:26 am

When iwlan runs on IOMMU, IOMMU generates a lot of PTE write faults because PTE
write bit is not set on some of PTE's. This is because iwlan driver calls DMA
mapping with PCI_DMA_TODEVICE which is read only in mapping PTE. But iwlan device
actually writes to the mapped page to update its contents. This issue is not
exposed in swiotlb. But VT-d hardware can capture this fault and stop the fault
transaction.

The following patch fixes the issue.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>

---

 iwl-tx.c |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

--- a/drivers/net/wireless/iwlwifi/iwl-tx.c.orig	2009-02-10 21:28:45.000000000 -0800
+++ b/drivers/net/wireless/iwlwifi/iwl-tx.c	2009-02-10 21:41:02.000000000 -0800
@@ -148,7 +148,7 @@ static void iwl_hw_txq_free_tfd(struct i
 		pci_unmap_single(dev,
 				pci_unmap_addr(&txq->cmd[index]->meta, mapping),
 				pci_unmap_len(&txq->cmd[index]->meta, len),
-				PCI_DMA_TODEVICE);
+				PCI_DMA_BIDIRECTIONAL);
 
 	/* Unmap chunks, if any. */
 	for (i = 1; i < num_tbs; i++) {
@@ -964,7 +964,7 @@ int iwl_tx_skb(struct iwl_priv *priv, st
 	 * within command buffer array. */
 	txcmd_phys = pci_map_single(priv->pci_dev,
 				    out_cmd, sizeof(struct iwl_cmd),
-				    PCI_DMA_TODEVICE);
+				    PCI_DMA_BIDIRECTIONAL);
 	pci_unmap_addr_set(&out_cmd->meta, mapping, txcmd_phys);
 	pci_unmap_len_set(&out_cmd->meta, len, sizeof(struct iwl_cmd));
 	/* Add buffer containing Tx command and MAC(!) header to TFD's
@@ -1115,7 +1115,7 @@ int iwl_enqueue_hcmd(struct iwl_priv *pr
 			IWL_MAX_SCAN_SIZE : sizeof(struct iwl_cmd);
 
 	phys_addr = pci_map_single(priv->pci_dev, out_cmd,
-				   len, PCI_DMA_TODEVICE);
+				   len, PCI_DMA_BIDIRECTIONAL);
 	pci_unmap_addr_set(&out_cmd->meta, mapping, phys_addr);
 	pci_unmap_len_set(&out_cmd->meta, len, len);
 	phys_addr += offsetof(struct iwl_cmd, hdr);
@@ -1212,7 +1212,7 @@ static void iwl_hcmd_queue_reclaim(struc
 	pci_unmap_single(priv->pci_dev,
 ...
From: Bhavesh Davda
Date: Wednesday, February 11, 2009 - 11:35 am

Looks good to me. Ship it!

- Bhavesh
 
--

From: Yu, Fenghua
Date: Wednesday, February 11, 2009 - 12:29 pm

Reviewed-by: Bhavesh Davda <bhavesh@vmware.com>

Thanks.

-Fenghua
--

From: Chris Wright
Date: Wednesday, February 11, 2009 - 11:59 am

Tested-by: Chris Wright <chrisw@sous-sol.org>

thanks,
chris
--

From: Winkler, Tomas
Date: Wednesday, February 11, 2009 - 2:27 pm

Acked-by: Tomas Winkler <tomas.winkler@intel.com>

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

--

From: Johannes Berg
Date: Wednesday, February 11, 2009 - 2:33 pm

-----BEGIN PGP SIGNATURE-----
Comment: Johannes Berg (powerbook)

iQIcBAABAgAGBQJJk0Q4AAoJEKVg1VMiehFYzpEP/R9SfF//dBafHymn63MOJ8gJ
pUjBtAqoRnRMJSZBU2M+AL+dfa9V4q+P+jDXU+WlI94/8TIoK85xaG2cejazgWtI
j6YzdeDRdRbJIY+PQ8D4F/pYOVQdVEelOuouE1d41kVXaFUwQxZwwRG5P5bfBNFi
y11COMXFubo5zltu8pjljQOlf/nP+GomGZxaM5k6fdpfoFxJW7XpI+Mht9a+juDR
oIn3OuBKnMcMwwRCRRRjmI2yGMHvJcLLm+7+wSUH0h5ltY8fG90iWdNfZSNafPDJ
+2DCwJaUT9MjYORdsqOxf61xxcdWR+bFPqb4vNhXsSqeAGUG5GyLdjiZQy+NU5B6
QjboTBwy8kJi9HHfbtxPYsk1QANpMQ3I465xrMNBhxi9itxl7HO3hickLf9DBfb6
KlCLtpYTwEjTLcTa5pI0/6NSne7boJ28fr2V6FhzUwsPLVbpNwVfeK3xWlxH119D
AjYvtckRuYGrGZEK72A9Wu6qiT/oCPIOjiDxfajPFXXeFXakM6cBFBi8CR2nOaAT
GHXMZ5todt17CCWCwmIG8nxIErgK5ISlke4kreXwb+n35VEcz5J+IwiWlzslFZE1
8uQTlhd23Ng1DEUAmZuvDi51pyZyXLw1CGL3BaA5rfMIDFq5lB50YXD8VuRWyfEp
o1eRECPd/QEHQK2wzPZS
=TW7w
-----END PGP SIGNATURE-----
From: Winkler, Tomas
Date: Wednesday, February 11, 2009 - 2:41 pm

Currently I'm not aware of any other cases where memory is accessed back but I may have not complete info.  Will be back with the answer tomorrow.
Tomas 

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

--

From: Johannes Berg
Date: Wednesday, February 11, 2009 - 2:47 pm

-----BEGIN PGP SIGNATURE-----
Comment: Johannes Berg (powerbook)

iQIcBAABAgAGBQJJk0dqAAoJEKVg1VMiehFY7yYQAKq8CiVHcI87LdoQ02/98i4c
kjsv/dCmWgJF1mWlpes2eh7ciNixUrluZq+56D9a4uduyi13ditUQ7ICu7Km9np2
sdyRNspC5KSe3bizpN5gzNQPnrZajDeSPrGjgM3CwxOtknORvXDssewXG7cRL8/H
w633+LxC297Kz6/00VQ6jfsE4aFJzTvHflFwiWfhw5POKRX1ubqqLiYKFCCqhCED
cA8rtoNKQ6hr1cyNEmHllobDzayk2w/W/r1SgIocrDO4GhIjh76CFkUG/NyeA4Y8
6W3MvL+v+rHMH76tFKOe1jbt5p4+IlpWT0zA4U6QOe4Go54aq2ByAu7AyboqJ03G
yyn2v+nw81oyz0upt7BJ+GrCY/TmUCXdu/1qtcXTcgA5AsbgHNZDKGo8nwYYUV6C
tJDZYoSmI3R/jOHl7sUZ8/jKETDaVRxte5dVojb6ss237PigN36q/APdZgX8g7xD
lXKavGsXw6BnlEHlJ16YSyn1ekDra3MUaMyrMY5jQ4ijpwCwrPSsECQO2N7JoCr0
J8DcHWSyqsk3xqqut3sjbb4I/p+srgKaPxYGlEgneorVI7x2+1/XXSQi7NaaiaiH
aNQeeqyXFQr9Hpz1A8XGI46x4ZmAkyUveiVKMb1Gb2+qH6Kg9Ti+gTUMXTBsHNFY
sXiKpFMHjGFnNXjsTjFu
=HYgZ
-----END PGP SIGNATURE-----
From: Johannes Berg
Date: Wednesday, February 11, 2009 - 2:50 pm

-----BEGIN PGP SIGNATURE-----
Comment: Johannes Berg (powerbook)

iQIcBAABAgAGBQJJk0gKAAoJEKVg1VMiehFYHzkQAI4TEGOISIDni20gm6Ca9oEE
2zqkUUI3rszUbh76rVCmGPmbSQCfvEI1fGr2sqszMqdcDeiPSvudxuSiMc7Dpt0G
I5ziz6gAL61ry1Do+qi3Ny3x+qs2eAcGDxx8Ua97NQvXrpuXx9ApH4ytvaFvu541
sbogJ9e46NwwLBOVyH3HgLHazX76uNQrunU6BdE+Z6qUS6XB9w36UNM/N2VTnVgb
rZj3q5oQ/xcElQXvw4fybXXZSGS+49qeUeQyJ1JEnoOPpHiOtUowu7RRnhqBtsmA
Bq1+FvEMvRGJVlrFHDVdKmrRCcl4qzglbW1iVoTIVHLpXWpDaKst31naf7f3w4ZO
fWPaMzqIV9kSwk/kPKeq1scxn5uXJC8A5yNlMljBbyGc5Zcpul8dFa4tqrFbTyai
wGG7d/g/TBZFZeJ9VVPHPbhf5vyNFyeZeTpSgOkslovx6uGZXUwQT5GhxPNaecas
gE2M9P6VLxEYWHjK2b0jgWDtv0Khp7WIwPul+pd88omARHf0YfwpbyMsSZcpJmYK
QTKBLICBAH/Y9bilBpcMndyAFT14NuJZgZQZTq5hNNUQHLHGlS5RKgSdJFDL1TES
TpMorbeXQyKCWQqmb3KgD5i/aUI3YfR+8LViTCBTiyTmNkSMo5RD0+yokGDeJN5s
mfyJ6FCP9YSJ9UvSJopf
=mlG3
-----END PGP SIGNATURE-----
From: Winkler, Tomas
Date: Wednesday, February 11, 2009 - 3:10 pm

That's why it's better to clear the picture. 
Fenghua
What exact HW do you get this error?

Thanks
Tomas 

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

--

From: Yu, Fenghua
Date: Wednesday, February 11, 2009 - 3:13 pm

This is Lenovo T400 wireless lan. Firmware version is 5.4.1.16. I think this issue happens on other platform/hw as well.

Thanks.

-Fenghua


--

From: David Woodhouse
Date: Wednesday, February 11, 2009 - 3:13 pm

Lenovo T400.

00:00.0 0600: 8086:2a40 (rev 07)
00:02.0 0300: 8086:2a42 (rev 07)
00:02.1 0380: 8086:2a43 (rev 07)
00:03.0 0780: 8086:2a44 (rev 07)
00:03.2 0101: 8086:2a46 (rev 07)
00:03.3 0700: 8086:2a47 (rev 07)
00:19.0 0200: 8086:10f5 (rev 03)
00:1a.0 0c03: 8086:2937 (rev 03)
00:1a.1 0c03: 8086:2938 (rev 03)
00:1a.2 0c03: 8086:2939 (rev 03)
00:1a.7 0c03: 8086:293c (rev 03)
00:1b.0 0403: 8086:293e (rev 03)
00:1c.0 0604: 8086:2940 (rev 03)
00:1c.1 0604: 8086:2942 (rev 03)
00:1c.2 0604: 8086:2944 (rev 03)
00:1c.3 0604: 8086:2946 (rev 03)
00:1c.4 0604: 8086:2948 (rev 03)
00:1d.0 0c03: 8086:2934 (rev 03)
00:1d.1 0c03: 8086:2935 (rev 03)
00:1d.2 0c03: 8086:2936 (rev 03)
00:1d.7 0c03: 8086:293a (rev 03)
00:1e.0 0604: 8086:2448 (rev 93)
00:1f.0 0601: 8086:2917 (rev 03)
00:1f.2 0106: 8086:2929 (rev 03)
00:1f.3 0c05: 8086:2930 (rev 03)
03:00.0 0280: 8086:4236 
04:00.0 0580: 8086:444e (rev 11)
15:00.0 0607: 1180:0476 (rev ba)
15:00.1 0c00: 1180:0832 (rev 04)

-- 
dwmw2

--

From: Winkler, Tomas
Date: Thursday, February 12, 2009 - 6:42 am

I can confirm that for 5000 series the patch is required also for regular frames. In this particular machine the NIC is 5300. For 4965 only for AMPDU frames are affected also the implementation is not the same but I'm not sure if it makes any sense to differentiate. 

Does anybody know what performance implications of this change are? 

Thanks Tomas  


---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

--

Previous thread: [PATCH] Rationalise email addresses by Alan Cox on Wednesday, October 1, 2008 - 9:53 am. (1 message)

Next thread: [PATCH 1/2]Add Variable Page Size and IA64 Support in Intel IOMMU: Generic Part by Fenghua Yu on Wednesday, October 1, 2008 - 9:57 am. (9 messages)