Subject: VC exceptions

From: Florian Lohoff <[email-masked]>
Date: Thu, 27 Apr 2000 16:58:03 +0200

Hi,
i had a conversation with Harald concerning a "strong" time drift
on my R4000 Decstation. He than was astonished on the large
number of VCE.

I than searched all my Mips books for an definition of those
exceptions. But i dont think i currently understand the 
cause of those exceptions.

BTW:

[flo@resume flo]$ uptime && cat /proc/cpuinfo 
  2:55pm  up 13 days, 20 min,  3 users,  load average: 1.15, 1.09, 1.01
  cpu                     : MIPS
  cpu model               : R4000SC V6.0
  system type             : SGI Indy
  BogoMIPS                : 124.93
  byteorder               : big endian
  unaligned accesses      : 90
  wait instruction        : no
  microsecond timers      : no
  extra interrupt vector  : no
  hardware watchpoint     : yes
  VCED exceptions         : 130546469
  VCEI exceptions         : 36073607

On a medium loaded machine i see 40-50 VCEDs per second.

Now i read in the "Mips R4000 Users`s Manual" page 133

------
Cause: A Virtual Coherency exception occurs when one of the
       following conditions is true:

       - a primary cache miss hits in the secondary cache
       - bits 14:12 of the virtual address were not equal to
         the corresponding bits of the PIdx field of the secondary 
	 cache tag.
       - the cache algorithm for the page specifies that the page is cached.
------

The "Mips Risc Architecture" says that ALL conditions are to be met which
i trust more :)

As a resume - The exception is taken when the index of the 1st and
the 2nd level cache are not identical - Right ?
So - why is there a mismatch ? Might it be due to some invalidation
of the 1st (and not the 2nd) level cache ?

As the exception is taken quiet often and the "Mips Risc Architecture" states
"Software can avoid the cost of this trap by using constistent virtual
primary cache indexes to access the same physical data".

Currently i dont think whats the exact cause of this exception and
a probably optimization which brings this down.

Flo
-- 
Florian Lohoff		[email-masked]		      	+49-subject-2-change
"Technology is a constant battle between manufacturers producing bigger and
more idiot-proof systems and nature producing bigger and better idiots."

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 29 Apr 2000 07:18:07 +0200

> On a medium loaded machine i see 40-50 VCEDs per second.

VCE isn't related to timekeeping and only may delay interrupts minimally;
they're very lightweight.  In fact keeping the VCE statistics makes a
significant part of the overhead.  So while the number of VCEs may look
high the total price is not too bad.

> primary cache indexes to access the same physical data".

Attached a small test program that generates a large number of vced exceptions.
It's causing aliases with the page cache and these are not covered by my
patch.

> a probably optimization which brings this down.

The easiest part of the solution is choosing apropriate addresses for all
memory mappings without forced addresses, that is mmap(2) and mmap2(2)
without MAP_FIXED.  A patch which fixes this part of the problem is attached.

This patch does not fix other types of aliases like mmap(2) and mmap2(2)
with MAP_FIXED or aliases between multiple mappings out of the page cache.

Users of R2000 / R3000 / R7000 / R10000 CPUs can ignore this whole thread,
those CPUs don't have such incredibly f*cked caches.

  Ralf

---

From: [email-masked]
Date: Sat, 29 Apr 2000 18:33:54 -0400 (EDT)

What is a r7000?  I've heard of the r8000, is that the same?
	Nick

---

From: Ralf Baechle <[email-masked]>
Date: Sun, 30 Apr 2000 00:45:58 +0200

> What is a r7000?  I've heard of the r8000, is that the same?

Stupid answer: No, otherwise they'd have the same name :-)

The R8000 was some kind of CPU hack which SGI came up with when the R4400
performance was begining to look bad in comparison to the Alphas and the
R10000 project still would have taken quite some time.  It was featuring
roughly the integer performance and twice or trice the fp performance of
a 250MHz R4400 while running at just 75 - 90 MHz.  It was used only by
SGI.

The R7000 is kind of a successor to the R5000 featuring roughly R10000
performance but at a much lower price.  This was developed by either
IDT or QED mostly for embedded purposes.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Sun, 30 Apr 2000 14:24:42 +0200

On Sun, Apr 30, 2000 at 11:56:03AM +0100, Dominic Sweetman wrote:

> So the R10000 was desperately complicated, and the RM7000 is simple
> but has onchip secondary cache.  At the same clock rate, the R10000 is
> going to be much faster (and use much more power, and cost much more
> to build into a system).  I doubt if even the just-announced 400Mhz
> RM7000 is really faster overall than a 180MHz R10000 - but Ralf might
> have access to some measurements.

I don't have hard numbers nor hardware, so I had to rely on the `hard facts'
that others had given to me.

In any case, the race stays unequal since the newest child of the R10000
family, the 400MHz R12000, will also enter the game soon if it isn't even
already shipping.

  Ralf