Subject: CONFIG_MIPS_UNCACHED

From: Jun Sun <[email-masked]>
Date: Sat, 23 Jun 2001 09:05:15 -0700

I looked at the code and it appears this config may not work properly.

My understanding is that if CPU has been running with cache enabled, and,
presummably, have many dirty cache entries, and if you suddenly change config
register to run kernel uncached, you *don't* get all the dirty cache lines
flushed into memory.  Therefore, you will be accessing stale data in memory.

Is this right?  If so, we need a better way to run CPU uncached.

In the past, I have been a private patch to do so.  It seems pretty difficult
to come up a generic, because we want to figure out the CPU type and disable
cache *before* kernel starts to modify any memory content.

BTW, this comes to me as I observe some weired behavior when I try to run
uncached.

Jun

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Sat, 23 Jun 2001 19:17:28 +0200

> cache *before* kernel starts to modify any memory content.

Since the kernel cache attribute is never initialized before
ld_mmu_{whatever} is invoked, and since that Config field
does not have a well-defined reset state on many MIPS
CPUs, it would appear that we are in effect trusting the
bootloader to have done something reasonable like
set kseg0 to be non-cachable or write-through, either
of which would be safe for the current code.  Strictly
speaking, if we do not wish to assume that the bootloader
has corectly initialized the caches, the kernel should begin
execution in kseg1and stay there until the caches have
been initialized and the kseg0 cache attribute has been
set to something sane.  And when I say "initialized", I
don't mean the writeback-invalidates used by the
blast_dcache routines, since those assume a sane
state of the tag ram, which cannot be guaranteed
at power-up.  One should write the tags directly to
a sane value on CPUs that support it  (on the R3000,
one apparently needs to write a byte to each cache
line to invalidate it.).

I would suggest either that we be explicit about the
assumptions about what the bootloader has done
to the cache, or we go whole-hog and assume that
the kernel has been branched to directly from the
reset vector.  Tinkering with measures in between
strikes me as a waste of time.

            Regards,

            Kevin K.

---

From: Jun Sun <[email-masked]>
Date: Sat, 23 Jun 2001 10:49:32 -0700

"Kevin D. Kissell" wrote:
> of which would be safe for the current code. 

I think you just proposed a fix: check current config register when we turn
off cache.  Thanks. :-)

Jun

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Sat, 23 Jun 2001 22:15:49 +0200

> I think you just proposed a fix: check current config register when we
turn

That's a heuristic at best.  If the config register comes up random,
it can appear to be sane even though the cache is in fact uninitialized.

            Kevin K.

---

From: Jun Sun <[email-masked]>
Date: Sun, 24 Jun 2001 21:42:32 -0700

> it can appear to be sane even though the cache is in fact uninitialized.

For any practical reasons, we can assume there is a loader for Linux,
and we can assume loader does not run with a random config register.

Jun

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Mon, 25 Jun 2001 08:51:01 +0200

> > > I think you just proposed a fix: check current config register
> > > when we turn off cache.  Thanks. :-)
> >

That's a position that would sound reasonable to someone working
on Linux for legacy DEC/SGI systems, but not one that I would
expect to satisfy someone working on embedded Linux.  It would
need to be governed by a config option, but I would think
that ultimately we need to have a Linux that can be ROMed
and branched to directly from the reset vector.  Why force
everyone doing an embedded MIPS/Linux widget to re-invent
the wheel?

            Kevin K.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 25 Jun 2001 15:22:11 +0200 (MET DST)

> I think you just proposed a fix: check current config register when we turn

 Note that many MIPS CPUs do not have the config register that could be
used to turn the cache off.  That's not a problem for the userland as it's
controlled on a page-by-page basis, but the kernel runs unmapped (except
from modules) and user vs kernel memory coherency problems arise.  I have
a patch that makes the kernel run in the KSEG1 space (which is uncached by
default even on processors that have the config register).  It needs a
minor clean-up for exception handlers, though, as they start in KSEG0 with
no possibility to override.  So they should jump to KSEG1 ASAP --
hopefully two icache lines are OK; if cache in non-functional then we are
screwed as using bootstrap exception vectors is not an option, usually.
Cacheability of KSEG0 may be further disabled if possible. 

 I'll send the patch soon, when I clean it up.

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: Jun Sun <[email-masked]>
Date: Mon, 25 Jun 2001 09:38:08 -0700

> everyone doing an embedded MIPS/Linux widget to re-invent

The currenct common practice in embedded world is:

1. during development stage, using a loader to download kernel to target.

2. during productization stage, use a separate rom loader to cold-start the
board and load the kernel from flash to RAM, assuming the kernel is on flash.

There are a couple of other vairants, but generally you do have a first stage
loader that will set up the environment right for Linux kernel.

Cold-starting a board and loading a kernel is highly board and system
specific.  Does not seem to make sense to get included in the kernel
structure.

Jun

---

From: "Justin Carlson" <[email-masked]>
Date: Mon, 25 Jun 2001 10:35:43 -0700

> everyone doing an embedded MIPS/Linux widget to re-invent

Because there are very good reasons for having a firmware seperate from
the OS.  Otherwise, you're more or less proposing a new run-time-environment
that has to do all the hardware sanitizations and initializations before we
get to the bootstrap environment.  That's going to be so system-specific and
disparate from the kernel that it doesn't, IMHO, make any sense to put it
there.  This is especially true since *not* having it in the kernel gives you
the chance to exploit the same firmware environment for non-linux OS'es. 

-Justin

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Tue, 26 Jun 2001 10:50:50 +0200

> the OS.

Yes, and there are also very good reasons for minimizing
the size and functionality of that bootloader.  One could have
a minimalist but functional MIPS bootloader that runs in kseg1
and hasn't the faintest idea what a cache is.  As I said in
an earlier message on this thread, we should either be explicit
about what we assume the bootloader will have done for us, or
prepare to have the relevant CPU/cache intitialization done
by the kernel.

> Otherwise, you're more or less proposing a new run-time-environment
> there.

Cache tag initialization is CPU-specific, not system specific.

>              This is especially true since *not* having it in the kernel
gives you
> the chance to exploit the same firmware environment for non-linux OS'es.

The systems I'm worried about don't *have* any non-Linux OSes.
I do not advocate unconditionally putting proper cache initialization
code into every MIPS/Linux kernel!  I wouldn't dream of preventing
some one else from putting their full faith in the perfectly understood
and well-documented bootloaders on their Indy or DECstation. ;-)
And people who have otherwise found it to be a reasonable design
trade off to build a cache-aware bootloader should not have to pay
the time or footprint to initialize the cache twice.

But so long as there are people who want to build new, specialized devices
running embedded Linux, it is in their interest that the MIPS/Linux kernel
distribution provide them with as much of the generic processor startup
functionality as possible, so that they can concentrate their energies
on making their products different and better instead of re-re-implementing
cache initialization code (and maybe getting it wrong).

But in any case, have no fear, I'm unlikely to be submitting
any such patch any time soon!

            Regards,

            Kevin K.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Tue, 26 Jun 2001 15:11:41 +0200 (MET DST)

> and well-documented bootloaders on their Indy or DECstation. ;-)

 With functions like malloc(), strlen(), strcpy(), printf(), getenv(),
etc. I wouldn't name the DECstation's firmware (the REX one) just a
bootloader.  It's more like a minimal libc with additional functionality
to interface to hw, it's very well documented indeed (as opposed to most
of DECstations' hw, sigh...) and it appears to present a kernel an
interface that conforms to published docs.

 Of course, I don't expect anyone to put a libc into their embedded
system's ROM.