Re: MMIO and gcc re-ordering issue

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Nick Piggin <nickpiggin@...>
Cc: Trent Piepho <tpiepho@...>, Russell King <rmk+lkml@...>, Benjamin Herrenschmidt <benh@...>, David Miller <davem@...>, <linux-arch@...>, <scottwood@...>, <linuxppc-dev@...>, <alan@...>, <linux-kernel@...>
Date: Tuesday, June 3, 2008 - 10:46 pm

On Wed, 4 Jun 2008, Nick Piggin wrote:

Hmm. Intel arch manyal, Vol 3, 10.3 (page 10-7 in my version):

  "If the WC bufer is partially filled, the writes may be delayed until 
   the next ocurrence of a serializing event; such as, an SFENCE or MFENCE 
   instruction, CPUID execution, a read or write to uncached memory, ..."

Any typos mine.

Anyway, Intel certainly seems to document that WC memory is serialized by 
any access to UC memory.

But yes, I can well imagine that AMD is different, and I also heartily 
would recommend rather being safe than sorry. Putting an explicit memory 
barrier in between those accesses when you know it might make a difference 
is just a good idea. 

But basically, as far as I know the thing was designed to be invisible to 
old software: that is the whole idea behind WC memory. So the design was 
certainly intended to be that you can generally mark a framebuffer-like 
structure WC without any software _ever_ caring, as long as you keep all 
control ports in UC memory.

Of course, because burst writes from the WC buffer are <i>so</i> much more 
efficient on the PCI bus than dribbling them out one write at a time, it 
didn't take long before all the graphics cards etc wanted to <i>also</i> 
mark their command queues as WC memory, so that you could burst out the 
commands to the ring buffers as fast as possible. So now you have both 
your frame buffer *and* your command buffers mapped WC, and now ordering 
really has to be ensured in software if you access both.

[ And then there are the crazy people who mark *main memory* as WC, 
  because they don't want to pollute the cache with all the data, and then 
  you have the issue of cache coherency etc crap. Which only gets worse 
  with SMP, especially if one processor thinks it has part of memory 
  exclusively cached, and another one - or even the same one, 
  through another aliasign address - ignores the cache protocol.

  And you now get unhappy CPU's that think that there is a bug in the 
  cache protocol and they get machine check faults.

  So what started out as a "we can do accesses to the frame buffer more 
  efficiently without anybody ever even having to know or care" has 
  turned into a whole nightmare of people using it for other things, and 
  then you very much _do_ have to care! ]

And it doesn't surprise me if AMD then didn't get exactly the same 
rules. 

Oh, well.

		Linus
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Tue May 20, 4:40 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Andreas Schwab, (Tue May 20, 6:00 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Tue May 20, 6:11 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Andreas Schwab, (Tue May 20, 6:47 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Tue May 20, 7:14 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Andreas Schwab, (Wed May 21, 4:03 am)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Wed May 21, 4:25 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Benjamin Herrenschmidt, (Tue May 20, 5:16 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Tue May 20, 6:00 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Benjamin Herrenschmidt, (Wed May 21, 10:00 am)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Wed May 21, 3:44 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Benjamin Herrenschmidt, (Wed May 21, 4:41 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Tue May 20, 6:55 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Benjamin Herrenschmidt, (Wed May 21, 10:01 am)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, David Miller, (Tue May 20, 6:39 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, David Miller, (Tue May 20, 6:53 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Benjamin Herrenschmidt, (Fri May 23, 12:24 am)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Thu May 22, 6:56 pm)
MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Mon May 26, 9:33 pm)
Re: MMIO and gcc re-ordering issue, Jonathan Corbet, (Tue May 27, 11:28 am)
Re: MMIO and gcc re-ordering issue, David Miller, (Mon May 26, 9:40 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Mon May 26, 10:15 pm)
Re: MMIO and gcc re-ordering issue, David Miller, (Mon May 26, 10:28 pm)
Re: MMIO and gcc re-ordering issue, Alan Cox, (Tue May 27, 4:24 am)
Re: MMIO and gcc re-ordering issue, Arjan van de Ven, (Mon May 26, 11:42 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 3:08 am)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 11:50 am)
Re: MMIO and gcc re-ordering issue, Pavel Machek, (Sat May 31, 4:04 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 5:11 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 5:33 pm)
Re: MMIO and gcc re-ordering issue, Jes Sorensen, (Thu May 29, 10:47 am)
Re: MMIO and gcc re-ordering issue, Jeremy Higdon, (Sat May 31, 3:52 am)
Re: MMIO and gcc re-ordering issue, Jes Sorensen, (Mon Jun 2, 5:56 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 12:33 am)
Re: MMIO and gcc re-ordering issue, Jesse Barnes, (Tue Jun 3, 12:52 pm)
Re: MMIO and gcc re-ordering issue, Jes Sorensen, (Thu Jun 5, 4:40 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Thu Jun 5, 4:43 am)
Re: MMIO and gcc re-ordering issue, Matthew Wilcox, (Thu Jun 12, 11:07 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Thu Jun 12, 8:07 pm)
Re: MMIO and gcc re-ordering issue, Jeremy Higdon, (Tue Jun 3, 4:15 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 4:19 am)
Re: MMIO and gcc re-ordering issue, Jeremy Higdon, (Tue Jun 3, 4:45 am)
Re: MMIO and gcc re-ordering issue, Jeremy Higdon, (Mon Jun 2, 5:02 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Thu May 29, 6:06 pm)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Thu May 29, 6:25 pm)
Re: MMIO and gcc re-ordering issue, Paul Mackerras, (Thu May 29, 11:56 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Thu May 29, 5:40 pm)
Re: MMIO and gcc re-ordering issue, Jes Sorensen, (Fri May 30, 5:48 am)
Re: MMIO and gcc re-ordering issue, Pavel Machek, (Sat May 31, 4:14 am)
Re: MMIO and gcc re-ordering issue, Jes Sorensen, (Mon Jun 2, 5:48 am)
Re: MMIO and gcc re-ordering issue, Jesse Barnes, (Thu May 29, 5:53 pm)
Re: MMIO and gcc re-ordering issue, Jes Sorensen, (Fri May 30, 5:39 am)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Thu May 29, 5:48 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Thu May 29, 6:05 pm)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Thu May 29, 9:53 pm)
Re: MMIO and gcc re-ordering issue, James Bottomley, (Thu May 29, 11:01 am)
Re: MMIO and gcc re-ordering issue, Jes Sorensen, (Fri May 30, 5:36 am)
Re: MMIO and gcc re-ordering issue, Jesse Barnes, (Fri May 30, 1:21 pm)
Re: MMIO and gcc re-ordering issue, Jeremy Higdon, (Sat May 31, 3:57 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 6:13 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 6:39 pm)
Re: MMIO and gcc re-ordering issue, James Bottomley, (Tue May 27, 12:37 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 1:38 pm)
Re: MMIO and gcc re-ordering issue, James Bottomley, (Tue May 27, 1:53 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 2:07 pm)
Re: MMIO and gcc re-ordering issue, Chris Friesen, (Tue May 27, 5:23 pm)
Re: MMIO and gcc re-ordering issue, Paul Mackerras, (Tue May 27, 7:04 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 5:29 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 2:17 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue May 27, 12:08 am)
Re: MMIO and gcc re-ordering issue, Arjan van de Ven, (Tue May 27, 12:20 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Mon May 26, 11:39 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue May 27, 11:35 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 5:10 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue May 27, 5:30 pm)
Re: MMIO and gcc re-ordering issue, Alan Cox, (Tue May 27, 5:38 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue May 27, 5:59 pm)
Re: MMIO and gcc re-ordering issue, Matthew Wilcox, (Tue May 27, 5:53 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue May 27, 6:02 pm)
Re: MMIO and gcc re-ordering issue, Alan Cox, (Tue May 27, 5:46 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 5:38 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue May 27, 5:55 pm)
Re: MMIO and gcc re-ordering issue, Russell King, (Mon Jun 2, 3:24 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 12:16 am)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Tue Jun 3, 3:43 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue Jun 3, 6:26 pm)
Re: MMIO and gcc re-ordering issue, Matthew Wilcox, (Tue Jun 3, 5:33 pm)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Tue Jun 3, 5:44 pm)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 10:25 pm)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Wed Jun 4, 2:39 am)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue Jun 3, 10:47 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 10:19 pm)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Tue Jun 3, 2:47 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue Jun 3, 3:07 pm)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 10, 2:56 am)
Re: MMIO and gcc re-ordering issue, Paul Mackerras, (Wed Jun 11, 1:20 am)
Re: MMIO and gcc re-ordering issue, Paul Mackerras, (Wed Jun 11, 12:18 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Wed Jun 11, 1:00 am)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Wed Jun 11, 10:46 am)
Re: MMIO and gcc re-ordering issue, Paul Mackerras, (Wed Jun 11, 1:13 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Wed Jun 11, 1:35 am)
Re: MMIO and gcc re-ordering issue, Paul Mackerras, (Thu Jun 12, 8:14 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Thu Jun 12, 9:08 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Wed Jun 11, 2:02 am)
Re: MMIO and gcc re-ordering issue, Jesse Barnes, (Tue Jun 10, 1:41 pm)
Re: MMIO and gcc re-ordering issue, James Bottomley, (Tue Jun 10, 2:10 pm)
Re: MMIO and gcc re-ordering issue, Roland Dreier, (Tue Jun 10, 3:05 pm)
Re: MMIO and gcc re-ordering issue, Jesse Barnes, (Tue Jun 10, 3:19 pm)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 10, 11:29 pm)
Re: MMIO and gcc re-ordering issue, Jesse Barnes, (Wed Jun 11, 12:07 pm)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Thu Jun 12, 7:27 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue Jun 10, 11:40 pm)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Wed Jun 11, 12:06 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 10:05 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue Jun 3, 10:46 pm)
Re: MMIO and gcc re-ordering issue, Alan Cox, (Wed Jun 4, 7:47 am)
Re: MMIO and gcc re-ordering issue, Matthew Wilcox, (Tue Jun 3, 2:55 pm)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Tue Jun 3, 3:57 pm)
Re: MMIO and gcc re-ordering issue, Matthew Wilcox, (Tue Jun 3, 5:35 pm)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Tue Jun 3, 5:58 pm)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 10:00 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue Jun 3, 12:32 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 2:11 am)
Re: MMIO and gcc re-ordering issue, Paul Mackerras, (Tue Jun 3, 2:53 am)
Re: MMIO and gcc re-ordering issue, Nick Piggin, (Tue Jun 3, 3:18 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue Jun 3, 2:48 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 6:19 pm)
Re: MMIO and gcc re-ordering issue, Arnd Bergmann, (Thu May 29, 3:10 am)
Re: MMIO and gcc re-ordering issue, Alan Cox, (Thu May 29, 6:46 am)
Re: MMIO and gcc re-ordering issue, Matthew Wilcox, (Tue May 27, 5:42 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 6:17 pm)
Re: MMIO and gcc re-ordering issue, Haavard Skinnemoen, (Wed May 28, 4:36 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Thu May 29, 9:13 pm)
Re: MMIO and gcc re-ordering issue, Haavard Skinnemoen, (Fri May 30, 2:07 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Fri May 30, 3:24 am)
Re: MMIO and gcc re-ordering issue, Haavard Skinnemoen, (Fri May 30, 4:27 am)
Re: MMIO and gcc re-ordering issue, Geert Uytterhoeven, (Fri May 30, 5:22 am)
Re: MMIO and gcc re-ordering issue, Haavard Skinnemoen, (Mon Jun 2, 4:11 am)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Wed Jun 4, 11:31 am)
Re: MMIO and gcc re-ordering issue, Scott Wood, (Mon Jun 2, 11:48 am)
Re: MMIO and gcc re-ordering issue, Haavard Skinnemoen, (Tue Jun 3, 3:46 am)
Re: MMIO and gcc re-ordering issue, Pantelis Antoniou, (Thu May 29, 7:05 am)
Re: MMIO and gcc re-ordering issue, Trent Piepho, (Tue May 27, 2:23 pm)
Re: MMIO and gcc re-ordering issue, Scott Wood, (Tue May 27, 2:33 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue May 27, 12:47 pm)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Tue May 27, 5:12 pm)
Re: MMIO and gcc re-ordering issue, Linus Torvalds, (Tue May 27, 1:31 pm)
Re: MMIO and gcc re-ordering issue, Ingo Molnar, (Mon Jun 2, 6:36 am)
Re: MMIO and gcc re-ordering issue, Benjamin Herrenschmidt, (Mon Jun 2, 5:53 pm)
MMIO and gcc re-ordering (Was: [PATCH] [POWERPC] Improve (in..., Benjamin Herrenschmidt, (Fri May 23, 8:36 am)
Re: MMIO and gcc re-ordering (Was: [PATCH] [POWERPC] Improve..., Benjamin Herrenschmidt, (Fri May 23, 8:50 am)
Re: MMIO and gcc re-ordering (Was: [PATCH] [POWERPC] Improve..., Benjamin Herrenschmidt, (Fri May 23, 6:47 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Benjamin Herrenschmidt, (Tue May 20, 6:02 pm)
Re: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code, Trent Piepho, (Tue May 20, 6:21 pm)