Subject: ...cache dimensioning ;-)

From: Emmanuel Michon <[email-masked]>
Date: Fri, 10 Sep 2004 12:15:58 +0200

Hi,

I'm still in the process of choosing the best configurable parameters of
a hardware design based on 4KEc

As far as I understand, excepted alpha platforms, 4KByte pages are the
de facto standard [I assume linux developers are reasonable so changing
the page size to 8KB is not going to be a nightmare...]

Since the mips cache is virtually indexed but physically tagged, I see
two problems when the size of a cache way exceeds the size of a page:

- virtual aliasing. Can only happen on R/W pages (data cache) and only
when two different virtual addresses map the same physical page. An
example of this is: two processes sharing a memory area; should I
consider this is taken into account by linux already?

- I was told the software exception handlers for tlb are much less
efficient when cacheway > pagesize, forcing to flush too often. Is this
true? What is, in practice, the ratio of instruction pages and data
pages in a tlb?

If I consider a platform like Toshiba TX39 which has d-cache four ways
with total 32KBytes, it must already have the problems above. I'd like
to get some more clues though...

Thanks a lot,

Sincerely yours,

E.M.

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 10 Sep 2004 13:43:43 +0200

> a hardware design based on 4KEc

Simply answer:

Linux is most happy if size_per_cache =< PAGE_SIZE * number_of_ways.

> the page size to 8KB is not going to be a nightmare...]

4k may be the predominant size but by no means it's the defacto standard.

Some older ARM chips use 16k or 32k; Cris uses 8k; some 68k Suns using
the non-Motorola MMU use 8k also; sun4 and sparc64 both use 8k; IA-64 is
configurable to 4k (bad idea), 16k (prefered) and 64k.

On MIPS going for 4k was my choice in '95 ago because that's the pagesize
of the R3000 - and the i386 which eleminated a whole class of bugs in
porting.

It's now ten years later; everything is growing according to Moore's law
and some embedded systems that are considered small by today's standards
exceed what was a powerful workstation back then or even a full blown
supermini in '85 when the R2000 was created so increasig the page size
is overdue for most systems, MIPS and other.

> two problems when the size of a cache way exceeds the size of a page:

Not all of them; it's just the most common cache policy.  There are also
PIPT caches and even VIVT I-caches.  I'll personally poke needles into
a doll of whoever should design VIVT D-caches ;-)

> true? What is, in practice, the ratio of instruction pages and data

The affected parts of memory managment are not part of the TLB exception
handlers in a strict sense but yes, on caches suffering from aliases in
situations where we can't use hit cacheops we need to flush nuber of ways
pages from the cache, so for a 4-way 32k cache that means half the cache.

Going to 16k pagesize would eleminate most such flushes entirely and as
a nice side effect also guarantee the correctness of a fairly complex
to get right piece of code by simple eleminating the need for it.

> to get some more clues though...

My paper says different numbers ...

  Ralf