Hi,

OK this was going to be a quick patch, but after sleeping on it, I think
it deserves a better analysis... I can prove the comment is incorrect with a
test program, but I'm not as sure about my thinking that leads me to call it
also misleading.


The comment being removed by this patch is incorrect and misleading (I think). 

1. load  ...
2. store 1 -> X
3. wmb
4. rmb
5. load  a <- Y
6. store ...

4 will only ensure ordering of 1 with 5.
3 will only ensure ordering of 2 with 6.

Further, a CPU with strictly in-order stores will still only provide that
2 and 6 are ordered (effectively, it is the same as a weakly ordered CPU
with wmb after every store).

In all cases, 5 may still be executed before 2 is visible to other CPUs!


The additional piece of the puzzle that mb() provides is the store/load
ordering, which fundamentally cannot be achieved with any combination of rmb()s
and wmb()s.

This can be an unexpected result if one expected any sort of global ordering
guarantee to barriers (eg. that the barriers themselves are sequentially
consistent with other types of barriers).  However sfence or lfence barriers
need only provide an ordering partial ordering of meomry operations -- Consider
that wmb may be implemented as nothing more than inserting a special barrier
entry in the store queue, or, in the case of x86, it can be a noop as the store
queue is in order. And an rmb may be implemented as a directive to prevent
subsequent loads only so long as their are no previous outstanding loads (while
there could be stores still in store queues).

I can actually see the occasional load/store being reordered around lfence on
my core2. That doesn't prove my above assertions, but it does show the comment
is wrong (unless my program is -- can send it out by request).

So:
mb() and smp_mb() always have and always will require a full mfence or lock
prefixed instruction on x86. And we should remove this comment.


[ This is true for x86's sfence/lfence, but raises a ...
[ message continues ]

You're 100% right, that comment is total crap.

An lfence is *not* a memory barrier, it's just a read memory barrier. 

Although on some microarchitectures they may of course end up being the 
same thing.

			Linus
-

[ message continues ]

No, that can't be. rmb+wmb can't be considered a full mb. There was a 
recent discussion about this in the thread originated by peterz scalable 
rw_mutex patches.



- Davide


-

[ message continues ]

Oh I realise it doesn't work with current implementations (powerpc at
least also would be broken I think). And I've always assumed no... But
the question was just whether the Linux memory model requires it (it
theoretically could require that smp_rmb is sequentially ordered WRT

OK, good to hear it has been discussed. I didn't see anything explicit
in the documentation about it.  A lot of the literature often says that
"barrier instructions" have a sequential ordering, so it could be useful
to explicitly say this isn't the case for Linux for rmb vs wmb.

-

[ message continues ]

Yes, because x86 allows loads to be executed before earlier stores,

Anyone that does expect this needs to adjust their expectations.  ;-)

-

[ message continues ]

-

[ message continues ]

Thanks for the detailed explanation. I queued the patch.

-Andi

-

[ message continues ]

I think you have to assume that smp_wmb() only orders stores and write
barriers, and that smp_rmb() only orders reads and read barriers.

smp_mb() implies both smp_wmb() and smp_rmb(), but is greater than the
combination of the two.

David
-

[ message continues ]