The following two patches (Intel IOMMU generic patch and ia64 specific patch) together enable Intel IOMMU on IA64 platform. They are applied cleanly on the latest linux-next tree which contains Intel IOMMU interrupt remapping and VT-d driver KVM support code. If applying only the generic patch on x86-64 kernel, x86-64 IOMMU is working fine, i.e. the generic patch doesn't have compilation and functionality impact on x86-64 IOMMU. If applying only the ia64 specific patch on ia64 kernel, ia64 is working as non-IOMMU/swiotlb case. If applying both of the patches and set CONFIG_DMAR=y, ia64 IOMMU is working. Thanks. -Fenghua --
General comment (not ia64-related): it looks like you detect the IOMMU using the static DMAR table. Does the IOMMU also appear in PCI config space or the ACPI namespace? I expect it would, because the fact that the DMAR table is static precludes any sort of hot-plug to add or remove IOMMUs. If it *is* in config space or the namespace, it would be good to exercise that discovery path to help shake out firmware bugs. For example, arch/ia64/hp/common/sba_iommu.c uses acpi_bus_register_driver() to discover HP IOMMUs. I wouldn't hold the sba_iommu.c discovery code (most of which I take the blame for) up as a shining example of how to do things, but it shows the point. Bjorn --
The latest VT-d spec v1.1 does not support hot plug. If the feature is available in newer spec, we will support it. Thanks. -Fenghua --
The patche set (Intel IOMMU generic patch and ia64 specific patch) together enable Intel IOMMU on IA64 platform. They are applied cleanly on the latest linux-next tree which contains Intel IOMMU interrupt remapping and VT-d driver KVM support code. If applying only the generic patch on x86-64 kernel, x86-64 IOMMU is working fine, i.e. the generic patch doesn't have compilation and functionality impact on x86-64 IOMMU. If applying only the ia64 specific patch on ia64 kernel, ia64 is working as non-IOMMU/swiotlb case. If applying both of the patches and set CONFIG_DMAR=y, ia64 IOMMU is working. Thanks. -Fenghua --
When iwlan runs on IOMMU, IOMMU generates a lot of PTE write faults because PTE
write bit is not set on some of PTE's. This is because iwlan driver calls DMA
mapping with PCI_DMA_TODEVICE which is read only in mapping PTE. But iwlan device
actually writes to the mapped page to update its contents. This issue is not
exposed in swiotlb. But VT-d hardware can capture this fault and stop the fault
transaction.
The following patch fixes the issue.
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
iwl-tx.c | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
--- a/drivers/net/wireless/iwlwifi/iwl-tx.c.orig 2009-02-10 21:28:45.000000000 -0800
+++ b/drivers/net/wireless/iwlwifi/iwl-tx.c 2009-02-10 21:41:02.000000000 -0800
@@ -148,7 +148,7 @@ static void iwl_hw_txq_free_tfd(struct i
pci_unmap_single(dev,
pci_unmap_addr(&txq->cmd[index]->meta, mapping),
pci_unmap_len(&txq->cmd[index]->meta, len),
- PCI_DMA_TODEVICE);
+ PCI_DMA_BIDIRECTIONAL);
/* Unmap chunks, if any. */
for (i = 1; i < num_tbs; i++) {
@@ -964,7 +964,7 @@ int iwl_tx_skb(struct iwl_priv *priv, st
* within command buffer array. */
txcmd_phys = pci_map_single(priv->pci_dev,
out_cmd, sizeof(struct iwl_cmd),
- PCI_DMA_TODEVICE);
+ PCI_DMA_BIDIRECTIONAL);
pci_unmap_addr_set(&out_cmd->meta, mapping, txcmd_phys);
pci_unmap_len_set(&out_cmd->meta, len, sizeof(struct iwl_cmd));
/* Add buffer containing Tx command and MAC(!) header to TFD's
@@ -1115,7 +1115,7 @@ int iwl_enqueue_hcmd(struct iwl_priv *pr
IWL_MAX_SCAN_SIZE : sizeof(struct iwl_cmd);
phys_addr = pci_map_single(priv->pci_dev, out_cmd,
- len, PCI_DMA_TODEVICE);
+ len, PCI_DMA_BIDIRECTIONAL);
pci_unmap_addr_set(&out_cmd->meta, mapping, phys_addr);
pci_unmap_len_set(&out_cmd->meta, len, len);
phys_addr += offsetof(struct iwl_cmd, hdr);
@@ -1212,7 +1212,7 @@ static void iwl_hcmd_queue_reclaim(struc
pci_unmap_single(priv->pci_dev,
...Looks good to me. Ship it! - Bhavesh --
Reviewed-by: Bhavesh Davda <bhavesh@vmware.com> Thanks. -Fenghua --
Tested-by: Chris Wright <chrisw@sous-sol.org> thanks, chris --
Acked-by: Tomas Winkler <tomas.winkler@intel.com> --------------------------------------------------------------------- Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --
-----BEGIN PGP SIGNATURE----- Comment: Johannes Berg (powerbook) iQIcBAABAgAGBQJJk0Q4AAoJEKVg1VMiehFYzpEP/R9SfF//dBafHymn63MOJ8gJ pUjBtAqoRnRMJSZBU2M+AL+dfa9V4q+P+jDXU+WlI94/8TIoK85xaG2cejazgWtI j6YzdeDRdRbJIY+PQ8D4F/pYOVQdVEelOuouE1d41kVXaFUwQxZwwRG5P5bfBNFi y11COMXFubo5zltu8pjljQOlf/nP+GomGZxaM5k6fdpfoFxJW7XpI+Mht9a+juDR oIn3OuBKnMcMwwRCRRRjmI2yGMHvJcLLm+7+wSUH0h5ltY8fG90iWdNfZSNafPDJ +2DCwJaUT9MjYORdsqOxf61xxcdWR+bFPqb4vNhXsSqeAGUG5GyLdjiZQy+NU5B6 QjboTBwy8kJi9HHfbtxPYsk1QANpMQ3I465xrMNBhxi9itxl7HO3hickLf9DBfb6 KlCLtpYTwEjTLcTa5pI0/6NSne7boJ28fr2V6FhzUwsPLVbpNwVfeK3xWlxH119D AjYvtckRuYGrGZEK72A9Wu6qiT/oCPIOjiDxfajPFXXeFXakM6cBFBi8CR2nOaAT GHXMZ5todt17CCWCwmIG8nxIErgK5ISlke4kreXwb+n35VEcz5J+IwiWlzslFZE1 8uQTlhd23Ng1DEUAmZuvDi51pyZyXLw1CGL3BaA5rfMIDFq5lB50YXD8VuRWyfEp o1eRECPd/QEHQK2wzPZS =TW7w -----END PGP SIGNATURE-----
Currently I'm not aware of any other cases where memory is accessed back but I may have not complete info. Will be back with the answer tomorrow. Tomas --------------------------------------------------------------------- Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --
-----BEGIN PGP SIGNATURE----- Comment: Johannes Berg (powerbook) iQIcBAABAgAGBQJJk0dqAAoJEKVg1VMiehFY7yYQAKq8CiVHcI87LdoQ02/98i4c kjsv/dCmWgJF1mWlpes2eh7ciNixUrluZq+56D9a4uduyi13ditUQ7ICu7Km9np2 sdyRNspC5KSe3bizpN5gzNQPnrZajDeSPrGjgM3CwxOtknORvXDssewXG7cRL8/H w633+LxC297Kz6/00VQ6jfsE4aFJzTvHflFwiWfhw5POKRX1ubqqLiYKFCCqhCED cA8rtoNKQ6hr1cyNEmHllobDzayk2w/W/r1SgIocrDO4GhIjh76CFkUG/NyeA4Y8 6W3MvL+v+rHMH76tFKOe1jbt5p4+IlpWT0zA4U6QOe4Go54aq2ByAu7AyboqJ03G yyn2v+nw81oyz0upt7BJ+GrCY/TmUCXdu/1qtcXTcgA5AsbgHNZDKGo8nwYYUV6C tJDZYoSmI3R/jOHl7sUZ8/jKETDaVRxte5dVojb6ss237PigN36q/APdZgX8g7xD lXKavGsXw6BnlEHlJ16YSyn1ekDra3MUaMyrMY5jQ4ijpwCwrPSsECQO2N7JoCr0 J8DcHWSyqsk3xqqut3sjbb4I/p+srgKaPxYGlEgneorVI7x2+1/XXSQi7NaaiaiH aNQeeqyXFQr9Hpz1A8XGI46x4ZmAkyUveiVKMb1Gb2+qH6Kg9Ti+gTUMXTBsHNFY sXiKpFMHjGFnNXjsTjFu =HYgZ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNATURE----- Comment: Johannes Berg (powerbook) iQIcBAABAgAGBQJJk0gKAAoJEKVg1VMiehFYHzkQAI4TEGOISIDni20gm6Ca9oEE 2zqkUUI3rszUbh76rVCmGPmbSQCfvEI1fGr2sqszMqdcDeiPSvudxuSiMc7Dpt0G I5ziz6gAL61ry1Do+qi3Ny3x+qs2eAcGDxx8Ua97NQvXrpuXx9ApH4ytvaFvu541 sbogJ9e46NwwLBOVyH3HgLHazX76uNQrunU6BdE+Z6qUS6XB9w36UNM/N2VTnVgb rZj3q5oQ/xcElQXvw4fybXXZSGS+49qeUeQyJ1JEnoOPpHiOtUowu7RRnhqBtsmA Bq1+FvEMvRGJVlrFHDVdKmrRCcl4qzglbW1iVoTIVHLpXWpDaKst31naf7f3w4ZO fWPaMzqIV9kSwk/kPKeq1scxn5uXJC8A5yNlMljBbyGc5Zcpul8dFa4tqrFbTyai wGG7d/g/TBZFZeJ9VVPHPbhf5vyNFyeZeTpSgOkslovx6uGZXUwQT5GhxPNaecas gE2M9P6VLxEYWHjK2b0jgWDtv0Khp7WIwPul+pd88omARHf0YfwpbyMsSZcpJmYK QTKBLICBAH/Y9bilBpcMndyAFT14NuJZgZQZTq5hNNUQHLHGlS5RKgSdJFDL1TES TpMorbeXQyKCWQqmb3KgD5i/aUI3YfR+8LViTCBTiyTmNkSMo5RD0+yokGDeJN5s mfyJ6FCP9YSJ9UvSJopf =mlG3 -----END PGP SIGNATURE-----
That's why it's better to clear the picture. Fenghua What exact HW do you get this error? Thanks Tomas --------------------------------------------------------------------- Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --
This is Lenovo T400 wireless lan. Firmware version is 5.4.1.16. I think this issue happens on other platform/hw as well. Thanks. -Fenghua --
Lenovo T400. 00:00.0 0600: 8086:2a40 (rev 07) 00:02.0 0300: 8086:2a42 (rev 07) 00:02.1 0380: 8086:2a43 (rev 07) 00:03.0 0780: 8086:2a44 (rev 07) 00:03.2 0101: 8086:2a46 (rev 07) 00:03.3 0700: 8086:2a47 (rev 07) 00:19.0 0200: 8086:10f5 (rev 03) 00:1a.0 0c03: 8086:2937 (rev 03) 00:1a.1 0c03: 8086:2938 (rev 03) 00:1a.2 0c03: 8086:2939 (rev 03) 00:1a.7 0c03: 8086:293c (rev 03) 00:1b.0 0403: 8086:293e (rev 03) 00:1c.0 0604: 8086:2940 (rev 03) 00:1c.1 0604: 8086:2942 (rev 03) 00:1c.2 0604: 8086:2944 (rev 03) 00:1c.3 0604: 8086:2946 (rev 03) 00:1c.4 0604: 8086:2948 (rev 03) 00:1d.0 0c03: 8086:2934 (rev 03) 00:1d.1 0c03: 8086:2935 (rev 03) 00:1d.2 0c03: 8086:2936 (rev 03) 00:1d.7 0c03: 8086:293a (rev 03) 00:1e.0 0604: 8086:2448 (rev 93) 00:1f.0 0601: 8086:2917 (rev 03) 00:1f.2 0106: 8086:2929 (rev 03) 00:1f.3 0c05: 8086:2930 (rev 03) 03:00.0 0280: 8086:4236 04:00.0 0580: 8086:444e (rev 11) 15:00.0 0607: 1180:0476 (rev ba) 15:00.1 0c00: 1180:0832 (rev 04) -- dwmw2 --
I can confirm that for 5000 series the patch is required also for regular frames. In this particular machine the NIC is 5300. For 4965 only for AMPDU frames are affected also the implementation is not the same but I'm not sure if it makes any sense to differentiate. Does anybody know what performance implications of this change are? Thanks Tomas --------------------------------------------------------------------- Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --
