Subject: PROPOSAL : multi-way cache support in Linux/MIPS

From: Jun Sun <[email-masked]>
Date: Tue, 01 Aug 2000 16:52:25 -0700

Ralf,

I have got NEC DDB5476 running stable enough that I am comfortable to
check in
my code.  Will you take it?

Assuming the answer is yes, there are several issues regarding checking
in.
I will bring them up one by one.

The first issue is multi-way cache support.  DDB5476 uses R5432 CPU
which
has two-way set-associative cache.  The problematic part is the
index-based cache operations in r4xxcache.h does not cover all ways in a
set.

I think this is a problem in general.  So far I have seen MIPS
processors with
2-way, 4-way and 8-way sets.  And I have seen them using ether least-
significant-addressing-bits or most-significant-addressing-bits
within a cache line to select ways.

Here is my proposal :

. introduce two variables,
        cache_num_ways - number of ways in a set
        cache_way_selection_offset - the offset added to the address to
select
                next cache line in the same set.  For LSBs addressing,
it is
                equal to 1.  For MSBs addressing, it is equal to
                cache_line_size / cache_num_ways.  (It can potentially
take
                care of some future weired way-selection scheme as long
as
                the offset is uniform)

. These variables are initialized in cpu_probe().

  (Alternatively, I think we should have cpu_info table, that contains
all
   these cpu information.  Then a general routine can fill in the based
on
   the cpu id.  This can get rid of a bunch of ugly switch/case
statements.)

. in the include/asm/r4kcache.h file, all Index-based cache operation
needs
  to changed like the following (for illustration only; need
optimization) :

-----
        while(start < end) {
                cache16_unroll32(start,Index_Writeback_Inv_D);
                start += 0x200;
        }
+++++
        while(start < end) {
                for (i=0; i< cache_num_ways; i++) {
                        cache16_unroll32(start +
i*cache_way_selection_offset,
                                         Index_Writeback_Inv_D);
                }
                start += 0x200;
        }
=====

What do you think?  If it is OK, I can do the patch.  The cpu_info table
is a nice wish, but I don't think I know enough to do that job yet.

Jun

---

From: Dominic Sweetman <[email-masked]>
Date: Wed, 2 Aug 2000 19:12:34 +0100 (BST)

Jun Sun ([email-masked]) writes:

> which has two-way set-associative cache.  The problematic part is
> the index-based cache operations in r4xxcache.h does not cover all
> ways in a set.
> 
> processors with 2-way, 4-way and 8-way sets.  And I have seen them
> using ether least- significant-addressing-bits or
> most-significant-addressing-bits within a cache line to select ways.

So far as I know the Vr5432 is the only CPU to do anything so silly as
using the lowest index bits to select the "way".  Certainly most CPUs
put the "way" bits above the cache-store-index; and MIPS now require
it to be done like that for MIPS32 and MIPS64 compatible parts.

The MS-selects-way organisation permits the cache to be initialised
without the software ever needing to know how many ways it has (just
crank the index up, but being careful about the tendency to recycle
the same way when pre-filling cache data).

Cache maintenance should always use "hit" type instructions, and you
don't need to know the cache organisation for those, even with the
Vr5432. 

I suggest you should implement the don't-care method, and then have a
cpu_info-driven special case for the unique and deprecated Vr5432.

Dominic Sweetman
Algorithmics Ltd
[email-masked]

---

From: Jun Sun <[email-masked]>
Date: Wed, 02 Aug 2000 14:38:52 -0700

Dominic Sweetman wrote:
> using the lowest index bits to select the "way". 

Actually Sony's R4500 uses the same low bits mechanism.

> don't need to know the cache organisation for those, even with the

Ideally - but no in reality.  Linux stills uses index-operations a lot.

Theorically, indexed flush is faster if the flushing are is bigger than
the cache size.

> cpu_info-driven special case for the unique and deprecated Vr5432.

If Vr5432 is really that unique, I think that is probably best way, at
least for now.

Jun