[rfc][patch 2/3] x86: fix IO write barriers

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Linux Kernel Mailing List <linux-kernel@...>, Linus Torvalds <torvalds@...>, Andi Kleen <ak@...>
Date: Thursday, October 4, 2007 - 1:22 am

wmb() on x86 must always include a barrier, because stores can go out of
order in many cases when dealing with devices (eg. WC memory).

Signed-off-by: Nick Piggin <npiggin@suse.de>

Index: linux-2.6/include/asm-i386/system.h
===================================================================
--- linux-2.6.orig/include/asm-i386/system.h
+++ linux-2.6/include/asm-i386/system.h
@@ -216,6 +216,7 @@ static inline unsigned long get_limit(un
 
 #define mb() alternative("lock; addl $0,0(%%esp)", "mfence", X86_FEATURE_XMM2)
 #define rmb() alternative("lock; addl $0,0(%%esp)", "lfence", X86_FEATURE_XMM2)
+#define wmb() alternative("lock; addl $0,0(%%esp)", "sfence", X86_FEATURE_XMM)
 
 /**
  * read_barrier_depends - Flush all pending reads that subsequents reads
@@ -271,18 +272,14 @@ static inline unsigned long get_limit(un
 
 #define read_barrier_depends()	do { } while(0)
 
-#ifdef CONFIG_X86_OOSTORE
-/* Actually there are no OOO store capable CPUs for now that do SSE, 
-   but make it already an possibility. */
-#define wmb() alternative("lock; addl $0,0(%%esp)", "sfence", X86_FEATURE_XMM)
-#else
-#define wmb()	__asm__ __volatile__ ("": : :"memory")
-#endif
-
 #ifdef CONFIG_SMP
 #define smp_mb()	mb()
 #define smp_rmb()	rmb()
-#define smp_wmb()	wmb()
+#ifdef CONFIG_X86_OOSTORE
+# define smp_wmb() 	wmb()
+#else
+# define smp_wmb()	barrier()
+#endif
 #define smp_read_barrier_depends()	read_barrier_depends()
 #define set_mb(var, value) do { (void) xchg(&var, value); } while (0)
 #else
Index: linux-2.6/include/asm-x86_64/system.h
===================================================================
--- linux-2.6.orig/include/asm-x86_64/system.h
+++ linux-2.6/include/asm-x86_64/system.h
@@ -159,12 +159,8 @@ static inline void write_cr8(unsigned lo
  */
 #define mb() 	asm volatile("mfence":::"memory")
 #define rmb()	asm volatile("lfence":::"memory")
-
-#ifdef CONFIG_UNORDERED_IO
 #define wmb()	asm volatile("sfence" ::: "memory")
-#else
-#define wmb()	asm volatile("" ::: "memory")
-#endif
+
 #define read_barrier_depends()	do {} while(0)
 #define set_mb(var, value) do { (void) xchg(&var, value); } while (0)
 
-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[rfc][patch 1/3] x86_64: fence nontemproal stores, Nick Piggin, (Thu Oct 4, 1:21 am)
[rfc][patch 3/3] x86: optimise barriers, Nick Piggin, (Thu Oct 4, 1:23 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Fri Oct 12, 4:25 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Linus Torvalds, (Fri Oct 12, 11:13 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Mon Oct 15, 3:44 am)
RE: [rfc][patch 3/3] x86: optimise barriers, David Schwartz, (Mon Oct 15, 10:38 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Nick Piggin, (Mon Oct 15, 4:09 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Mon Oct 15, 5:10 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Nick Piggin, (Mon Oct 15, 8:50 pm)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Tue Oct 16, 5:00 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Tue Oct 16, 8:49 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Mon Oct 15, 5:24 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Nick Piggin, (Fri Oct 12, 4:57 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Fri Oct 12, 5:55 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Nick Piggin, (Fri Oct 12, 6:42 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Fri Oct 12, 7:55 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Fri Oct 12, 8:10 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Helge Hafting, (Fri Oct 12, 4:42 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Fri Oct 12, 5:12 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Helge Hafting, (Fri Oct 12, 8:44 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Fri Oct 12, 9:29 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Helge Hafting, (Mon Oct 15, 6:17 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Mon Oct 15, 7:53 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Nick Piggin, (Fri Oct 12, 5:44 am)
Re: [rfc][patch 3/3] x86: optimise barriers, Jarek Poplawski, (Fri Oct 12, 6:04 am)
[rfc][patch 2/3] x86: fix IO write barriers, Nick Piggin, (Thu Oct 4, 1:22 am)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Dave Jones, (Thu Oct 4, 1:32 pm)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Andi Kleen, (Thu Oct 4, 1:53 pm)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Dave Jones, (Thu Oct 4, 2:10 pm)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Andi Kleen, (Thu Oct 4, 2:21 pm)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Dave Jones, (Thu Oct 4, 2:41 pm)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Andi Kleen, (Thu Oct 4, 2:58 pm)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Dave Jones, (Thu Oct 4, 3:08 pm)
Re: [rfc][patch 2/3] x86: fix IO write barriers, Alan Cox, (Thu Oct 4, 4:52 pm)