Subject: CONFIG_MIPS_UNCACHED

From: Quinn Jensen <[email-masked]>
Date: Tue, 23 Jan 2001 16:26:35 -0700

Ralf,

On some machines with weird firmware (e.g. IDT 334 board)
the processor comes up with the cache already enabled for
kseg0.  In this case, the set_cp0_config() call in mips32.c
to turn off the cache (gated by CONFIG_MIPS_UNCACHED) should
probably come after the first call to flush_cache_all(),
which is safer but still not totally safe, I suppose.
Or am I totally hosed trying to turn the kseg0 cache off
after it was once on?

Quinn Jensen

---

From: Pete Popov <[email-masked]>
Date: Tue, 23 Jan 2001 15:53:27 -0800

> Or am I totally hosed trying to turn the kseg0 cache off

That's an issue not only when you're "turning off" the cache, but
whenever you muck with the kseg0 cache coherency attribute.  The Galileo
EV96100, running Galileo's pmon, comes up with kseg0 set to 3, which is
the default linux kseg0 cache coherency attribute. However, calling
set_cp0_config() without first flushing the cache destroys some data,
eventhough the same exact kseg0 attribute is set. 

I think a complete cache flush is in order before calling
set_cp0_config() and that's a change we should make for all cpus.

Pete

---

From: Jun Sun <[email-masked]>
Date: Wed, 24 Jan 2001 12:10:32 -0800

> eventhough the same exact kseg0 attribute is set.

It is really surprising to know this.  It sounds like a CPU bug to me.  Can
some MIPS "gods" clarify if such a behaviour is a bug or allowed?

BTW, the CPU in EV96100 is QED RM7000, I believe.

Jun

---

From: Ralf Baechle <[email-masked]>
Date: Wed, 24 Jan 2001 16:31:02 -0800

> BTW, the CPU in EV96100 is QED RM7000, I believe.

If you want to be strictly correct you have to execute the code that
disables caching of KSEG0 in uncached space like KSEG1, then flush the
caches before you can resume execution in KSEG0.  Otherwise you might
end up with dirty d-caches which when flushed will overwrite more
uptodate data in memory.  The window is very small but yet exists if
things are just right.

  Ralf

---

From: Pete Popov <[email-masked]>
Date: Wed, 24 Jan 2001 17:09:12 -0800

> uptodate data in memory.  The window is very small but yet exists if

The EV96100 running Galileo's pmon exhibits exactly this symptom.  Pmon
apparently sets up kseg0 to cache coherency 3;  but eventhough the
kernel also sets it to 3, if I don't flush the caches first I end up
with overwritten data.  A different version of pmon that I have sets
kseg0 to 1 (writethrough). Changing that to 3 isn't a problem -- or at
least it doesn't seem to cause any problems.

Pete

---

From: Jun Sun <[email-masked]>
Date: Wed, 24 Jan 2001 17:37:31 -0800

> least it doesn't seem to cause any problems.

I don't think it is the same problem.

Here is the simplified view of the process, if I understand Pete correctly:

1. pmon sets kseg0 to 3 (cache enabled)
2. kernel starts in KSEG0 
3. kernel sets kseg0 to 3 again (essentially keeps the same config value)
4. kernel flushes cache
  ===> Q: data corruption or not?

I think the data should be consistent.  Otherwise it looks like a CPU bug to
me.

What ralf described is something like the following:

1. pmon sets kseg0 to 3 (KSEG0 cache enabled)
2. kernel starts in KSEG0 
3. kernel sets kseg0 to 2 (disable kseg0 cache)
4. kernel flushes cache
  ===> Q: data corruption or not?  YES, data can be corrupted!

Jun

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Fri, 26 Jan 2001 11:02:24 +0100

> uptodate data in memory.  The window is very small but yet exists if

That's one possible failure mode.   The other is that there
are dirty lines (those that were brought into the cache and
modified by stores, but never written back to memory) that
are rendered invisible by forcing KSEG0 to go uncached.
This will cause failure of any code that is expecting to see
the results of those stores if said code executes before
some other event causes the write-back - and if the
machine is running with cacheing completely disabled,
as it would be in the case under discussion, such an
event may never occur!

            Regards,

            Kevin K.