Subject: Re: RM7k cache_flush_sigtramp

From: Fuxin Zhang <[email-masked]>
Date: Fri, 01 Aug 2003 08:40:14 +0800

Adam Kiepul wrote:

>Hi,
>
>If this is just to ensure the I Cache coherency for modified code then the following should be sufficient:
>
>cache Hit_Writeback_D, offset(base_register)
>cache Hit_Invalidate_I, offset(base_register)
>  
>
Current linux code does exactly this. But I was seeing all kinds of 
faults occuring around the
sigreturn point on the stack without a sync? And a sync does greatly 
improve the stablity.

>The ordering does matter however since the Hit_Invalidate_I makes sure the write buffer is flushed.
>
>Kind Regards,
>
>_______________________________
>
>Adam Kiepul
>Sr. Applications Engineer
>
>PMC-Sierra, Microprocessor Division
>Mission Towers One
>3975 Freedom Circle
>Santa Clara, CA 95054, USA
>Direct: 408 239 8124
>Fax: 408 492 9462
>
>
>
>-----Original Message-----
>From: Ralf Baechle [mailto:[email-masked]]
>Sent: Thursday, July 31, 2003 4:47 AM
>To: Fuxin Zhang
>Cc: MAKE FUN PRANK CALLS
>Subject: Re: RM7k cache_flush_sigtramp
>
>
>On Thu, Jul 31, 2003 at 09:56:08AM +0800, Fuxin Zhang wrote:
>  
>
>>Date:	Thu, 31 Jul 2003 09:56:08 +0800
>>From:	Fuxin Zhang <[email-masked]>
>>To:	MAKE FUN PRANK CALLS <[email-masked]>
>>    
>>
>        ^^^^^^^^^^^^^^^^^^^^
>
>Funny name for the list :-)
>
>  
>
>>r4k_cache_flush_sigtrap seems not enough for RM7000 cpus because
>>there is a writebuffer between L1 dcache & L2 cache,so the written back
>>block may not be seen by icache. This small patch fixes crashes of my
>>Xserver on ev64240.
>>    
>>
>
>It would seem a similar fix is also needed in other places then?
>
>  Ralf
>
>
>  
>

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 1 Aug 2003 05:01:11 +0200

Adam,

> >The ordering does matter however since the Hit_Invalidate_I makes sure the 

could there be an errata explaining Fuxin's findings?

Fuxin, what version are you running?

  Ralf

---

From: Fuxin Zhang <[email-masked]>
Date: Fri, 01 Aug 2003 12:59:00 +0800

I am using a slightly modified 2.4.21-pre4,based on cvs of early this 
month(?).
We have merged with latest cvs, I will run it and report the result tonight.

---

From: Dominic Sweetman <[email-masked]>
Date: Fri, 1 Aug 2003 08:51:39 +0100

> sure the write buffer is flushed.

I'm probably jumping into the middle of something, sorry... 

The MIPS32/MIPS64 release 2 architecture includes a useful instruction
SYNCI which does the whole job (repeat on each affected cache line)
and is legal in user mode; this will take a while to spread but I'd
recommend it as a model worth following.

So I hope that kernels will provide one function for "I've just
written instructions and now I want to execute them", and not export
the separate writeback-D/invalidate-I interface.

--
Dominic Sweetman
MIPS Technologies
[email-masked]

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 1 Aug 2003 11:26:49 +0200

> the separate writeback-D/invalidate-I interface.

Linux supports the traditional MIPS UNIX cacheflush(2) syscall through
a libc interface.  Since I've not seen any other use for the call than
I/D-cache synchronization.  I'd just make cacheflush(3) use SYNCI where
available (Or maybe one of the other vendor specific mechanisms ...) and
fallback to cacheflush(2) where available.  Gcc would be another place
to teach about SYNCI for it's trampolines.

  Ralf

---

From: Fuxin Zhang <[email-masked]>
Date: Fri, 01 Aug 2003 22:18:50 +0800

I just run a fresh new 2.4.21 kernel on my board, no luck.  The problem 
remains.
But I notice that my hardware may have some problems,especially with the 
add-on
ide card. Keep headaching...

As to the discussion of SYNC, I can't help wondering whether the cache 
management
should be totally hidden from programmers. People tends to write 
"safetest" code because
of all kinds of brain-damage different hardware, which leads to 
inefficient code. And this will
cancel out the potential speed benefit of simpler hardware. Also today's 
hardware seems not
as expensive as it was before...

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 2 Aug 2003 19:02:46 +0200

On Fri, Aug 01, 2003 at 10:18:50PM +0800, Fuxin Zhang wrote:

> remains.  But I notice that my hardware may have some problems,
> especially with the add-on ide card. Keep headaching...
> 
> management should be totally hidden from programmers. People tends to
> write "safetest" code because of all kinds of brain-damage different
> hardware, which leads to inefficient code. And this will cancel out the
> potential speed benefit of simpler hardware. Also today's hardware seems
> not as expensive as it was before...

Cache managment needs to be somehow hidden from programmers as well as
possible - the average programmer has no clue about how caches work.
We've come up with an API that hides the actual functioning of caches
pretty well for DMA devices, see Documentation/DMA-mapping.txt and in
2.6 also a more generalized version documented in Documentation/DMA-API.txt.

  Ralf

---

From: Dominic Sweetman <[email-masked]>
Date: Mon, 4 Aug 2003 09:45:33 +0100

Ralf,

> available...

SYNCI just does what's required to execute code you just wrote: that's
a D-cache writeback and an I-cache invalidate.  It doesn't invalidate
the D-cache afterwards, which is required by the definition of
cacheflush(3).

I think it would be better to provide cache manipulation calls defined
top-down (by their purpose); but so long as we are stuck with calls
which are defined as performing particular low-level actions, it's
surely dangerous to guess that we know what they are used for so we
can trim the functions accordingly...

--
Dominic Sweetman
MIPS Technologies

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 4 Aug 2003 13:51:57 +0200 (MET DST)

Dominic,

> can trim the functions accordingly...

 The API is not cast in stone -- if there's a justifiable benefit,
appropriate fuctions can be added; either completely new ones (possibly
inlined) or as an extension to cacheflush() (which still has 30 bits
freely available). 

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +