Subject: MC Parity Error

From: Florian Lohoff <[email-masked]>
Date: Fri, 23 Apr 2004 10:02:48 +0200

Hi,
success report for the MC Bus Error handler :)

Apr 19 23:17:32 resume kernel: MC Bus Error
Apr 19 23:17:32 resume kernel: CPU error 0x380<RD PAR > @ 0x0f4c6308
Apr 19 23:17:32 resume kernel: Instruction bus error, epc == 2accf310, ra == 2accf2c8

I guess i have bad memory. The interesting point is that the machine
continued to run for another 2 days. Shouldnt a memory error halt the
machine ?

Flo
-- 
Florian Lohoff                  [email-masked]             +49-171-2280134
                        Heisenberg may have been here.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Fri, 23 Apr 2004 15:11:19 +0200 (CEST)

> continued to run for another 2 days. Shouldnt a memory error halt the

 As it happened in the user mode, I'd expect only the victim process to be
killed.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 23 Apr 2004 18:45:17 +0200

>  As it happened in the user mode, I'd expect only the victim process to be

The KSU bits are meaningless.  On Indy like most other MIPS systems a
bus error exception may be delayed.  So the generic solution requires
tracking down the actual user, something which in the current kernel is
relativly easy due to rmap.

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Fri, 23 Apr 2004 19:11:36 +0200 (CEST)

> bus error exception may be delayed.  So the generic solution requires

 I beg your pardon?  AFAIK, bus errors are documented to be reported
precisely and my past experience with the systems I use confirms this.  
Otherwise bits in <asm/paccess.h> wouldn't work, but they do.  Of course
this is true for errors happening on read transactions (I have troubles
imagining a delayed read), but the semantics of the exception is defined
only for reads anyway.  For other transactions a general-purpose interrupt
should be used (and normally is).  Such an interrupt can happen any time,
indeed (but here it was an IBE, not an interrupt).

> relativly easy due to rmap.

 Well, that may be tough anyway -- imagine an uncorrectable memory error 
on a DMA transaction. ;-)

---

From: Dominic Sweetman <[email-masked]>
Date: Fri, 23 Apr 2004 13:43:42 -0700

Maciej W. Rozycki ([email-masked]) writes:

> precisely...

You're both right :-) Data errors like this on an R4x00 are reported
as cache parity errors, and cache parity error exceptions are precise.
There's also a signalling mechanism typically used for an invalid
memory address, which generates a "bus error" exception, which is not
precise.

--
Dominic Sweetman
MIPS Technologies.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Fri, 23 Apr 2004 23:06:24 +0200 (CEST)

> memory address, which generates a "bus error" exception, which is not

 I refer to the situation, when SysCmd(5) is set in a response to a
processor read request.

---

From: Stanislaw Skowronek <[email-masked]>
Date: Sat, 24 Apr 2004 08:28:09 +0200 (MET DST)

Hello,

why do we attempt to compile the kernel with 32-bit GAS abi and 64-bit GCC
abi? Is it because the module loader is broken and supports only 32-bit
ELFs? Then what about machines which load their kernels at weird 64-bit
addresses, like 0xa800000020004000 (Octane)?

I have changed it to 64-bit abi in my Octane kernel, because it won't even
compile otherwise. I've got gcc 3.3.2, gas 2.14.

Stanislaw Skowronek

--<=>--
  Paranoid: one who is truly in touch with reality.

---

From: Thiemo Seufer <[email-masked]>
Date: Sat, 24 Apr 2004 09:09:07 +0200

Stanislaw Skowronek wrote:
> abi?

It optimizes away a few hundred kB of kernel code, but requires in turn
a sign-extended load-address plus ugly objcopy hacks.

> Is it because the module loader is broken and supports only 32-bit

Ah, the same as for IP28. :-) They can't be supported by the current
scheme.

> compile otherwise. I've got gcc 3.3.2, gas 2.14.

You'll have to extend all the hand-coded asm memory accesses to do
64bit adressing as well.

Thiemo

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Apr 2004 09:17:52 +0200

> addresses, like 0xa800000020004000 (Octane)?

The whole thing was born as a dirty hack back in '99 or so to avoid the
totally broken and imcomplete implementation of 64-bit MIPS ELF in binutils.
By using these options we run the kernel in CKSEG0 which means 32-bit ELF
will suffice.  A nice side effect if a reduction of kernel size - this in
the end made the code model which was born as a hack the way of choice.

So there is no relation at all to modules.  You btw. can load 64-bit ELF
modules into a kernel which was built using above trick as 32-bit ELF.
That's necessary because modules are currently allocated through vmalloc
which allocates space in XKSEG.

A code model which I'm considering as alternative is -G optimization but
that will require a good bit of work; we would have to free up $28 which
right now we're already using for the current pointer.  A bunch of
declaration would simply need to be fixed.  Module loaders would need
support for R_MIPS_GPREL16 and we'd need a solution for the problem of
the kernel being too large, therefore overflowing the small data section.
Th latter is probably going to require binutils hacking, so non-trivial.

> compile otherwise. I've got gcc 3.3.2, gas 2.14.

Octane has no memory at all in CKSEG0?

>   Paranoid: one who is truly in touch with reality.

The fact that you're paranoid doesn't meant they're not out to get you.

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Sat, 24 Apr 2004 09:18:00 +0200 (CEST)

> addresses, like 0xa800000020004000 (Octane)?

1. Backward compatibility.  Old versions of gas/ld were buggy or
non-functional (depending on the version used) when using the (n)64 ABI.  
Search the mailing list archives -- I'm pretty sure anything since
2.13.2.1 should be safe, though.

2. Using the o32 ABI makes the binary smaller due to 32-bit pointers.  If
used without care, it can lead to pointer crops, though.  Anyway, some
people say it's important for them, despite the associated hassle.

> compile otherwise. I've got gcc 3.3.2, gas 2.14.

 I know.  I build using (n)64 consistently for two years successfully --
it's OK even with gcc 2.95.x.  Making a choice between the ABIs for gas
user-selectable is on my to-do list for some time.  For now I think `make
gas-abi=64 ...' is probably the easiest workaround, though you'll need to
objcopy the resulting image to a 32-bit ELF file manually if your firmware
or loader cannot cope with 64-bit ELF binaries.  Well, I don't like the
automatic copy anyway -- it wastes too much disk space in the long run;
perhaps as a compromise it should be user-selectable, too (ditto about
SREC).

  Maciej

---

From: Stanislaw Skowronek <[email-masked]>
Date: Sat, 24 Apr 2004 09:31:14 +0200 (MET DST)

> which allocates space in XKSEG.

Ah, so it's like that. Great. Is the ELF64 support still not correct?

> Octane has no memory at all in CKSEG0?

Well, as far as I know, and I'm probably right, it _does_ have some memory
there. A whopping 16 kilobytes of memory mirrored by the HEART to allow
placing exception vectors there (what a weird idea).

And you can't remap, I repeat, can't remap anything under 0x20000000
because there are the small Xtalk windows and HEART registers and God only
knows what else.

Stanislaw Skowronek

---

From: Stanislaw Skowronek <[email-masked]>
Date: Sat, 24 Apr 2004 09:34:55 +0200 (MET DST)

> perhaps as a compromise it should be user-selectable, too (ditto about

True, the kernel is *huge* (some 7 MB). But there *will* be pointer crops
if I'm using the xkphys, and I can't use ckseg0 because there are only 16
kilobytes of RAM mapped there for exceptions. So I have to use abi=64. It
does work for me, anyway.

I think it really should be a config option. Even if not actually
user-selectable (why should it be?), it should default to 'y' for Octanes
and 'n' for everything else :)

Thank you for all your explanations. My idea about modules was because I
noticed some int->ptr conversion warnings I don't like at all.

Stanislaw Skowronek

---

From: Thiemo Seufer <[email-masked]>
Date: Sat, 24 Apr 2004 09:37:03 +0200

Ralf Baechle wrote:
[snip]
> right now we're already using for the current pointer.

Gas has currently not much support for non-PIC n64 GP optimizations. The
recent changes have simplified the task of implementing them, though.

Thiemo

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Apr 2004 09:38:02 +0200

> Ah, so it's like that. Great. Is the ELF64 support still not correct?

No, it's supposed to be working now.

> placing exception vectors there (what a weird idea).

That's what the processor expects.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Apr 2004 09:44:07 +0200

> does work for me, anyway.

You could use a mapped address space, CKSEG2/3.

  Ralf

---

From: Stanislaw Skowronek <[email-masked]>
Date: Sat, 24 Apr 2004 09:46:46 +0200 (MET DST)

> No, it's supposed to be working now.

OK. File it away under 'compatibility cruft' then ;)

> That's what the processor expects.

Yeah. The weirdness is not in that part; what's weird is placing the rest
of memory somewhere else.

Stanislaw

---

From: Stanislaw Skowronek <[email-masked]>
Date: Sat, 24 Apr 2004 09:47:56 +0200 (MET DST)

> You could use a mapped address space, CKSEG2/3.

Yes, I could. I even know how to do that (thanks to Ralf's IP27 support),
however I think that making the kernel 64-bit-clean would benefit us all.

Stanislaw

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Sat, 24 Apr 2004 09:48:58 +0200 (CEST)

> True, the kernel is *huge* (some 7 MB). But there *will* be pointer crops

 Hmm, it depends on the number of drivers built-in -- my binaries are
usually around 3MB.  Or perhaps you should try strip? ;-)  You'll lose
symbol information for ksymoops then, though.

> does work for me, anyway.

 Sure, why wouldn't it?  Note, the current Makefile setup is intentionally 
easily tweakable for a fully 64-bit build.

> and 'n' for everything else :)

 Well, I don't need it to be selectable, but I'd set all systems to use
the (n)64 ABI unconditionally, and the next minute someone would complain.  
;-)  Perhaps it can be unconditional for systems where the other setting 
makes no sense at all.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Sat, 24 Apr 2004 09:51:22 +0200 (CEST)

> of memory somewhere else.

 Perhaps to be able to put iomem stuff in CKSEG1 without implying a hole
in the RAM.

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Apr 2004 09:55:45 +0200

>  Perhaps to be able to put iomem stuff in CKSEG1 without implying a hole

The machine is running a 64-bit kernel so likely it was designed with
little consideration for 32-bit issues.

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Sat, 24 Apr 2004 10:07:02 +0200 (CEST)

> little consideration for 32-bit issues.

 Well, the exception arrangement requires RAM starting from the physical
address 0.  It seems natural to place RAM just there, avoiding additional
complexity to address decoders.  But then firmware has to be somewere
around 0x1fc00000, so to support more than 508MB of RAM the designers
would have to create a hole in RAM, which would have to be handled by the
OS then.  Thus abandoning the idea of putting RAM low, placing it
somewhere above 0x1fffffff and just mapping some of it at 0 for the
exceptions seems a better solution.

 Fortunately everything is not a PC. :-)

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Apr 2004 10:14:05 +0200

> OK. File it away under 'compatibility cruft' then ;)

The size difference this makes is still very significant.  In case of an
IP27 kernel default config:

   text    data     bss     dec     hex filename
2626662  747232  165760 3539654  3602c6 vmlinux
2907645 1283808  165760 4357213  427c5d vmlinux

The first kernel was built with the stock Makefile; the second was modified
to use 64-bit ELF using gcc 2.95.4 / binutils 2.13.2.1.  So I'd call those
817559 bytes kernel obesity ;)

> of memory somewhere else.

Not uncommon on SGI systems.  The Indy's memory also starts at 128MB; only
a few kB for exeption vectors are mirrored to physical address 0.

  Ralf

---

From: Stanislaw Skowronek <[email-masked]>
Date: Sat, 24 Apr 2004 10:14:13 +0200 (MET DST)

> exceptions seems a better solution.

OK, I forgot about the firmware placement. Why didn't they move it to
somewhere else when booting 64-bit? (A rhetorical question, I know.)

I would place some fixed code there. Or a different memory, maybe 16 kB of
static RAM so it will always be fast. With what is now, we have physical
and virtual aliasing and it's all a bit like a 

> Fortunately everything is not a PC. :-)

Yes, fortunately. The 386 memory management is a joke. The BIOS in the
middle is an even darker joke. Well, in my opinion the R8000 was right in
not having compatibility segments.

Stanislaw

---

From: Stanislaw Skowronek <[email-masked]>
Date: Sat, 24 Apr 2004 10:17:42 +0200 (MET DST)

> 817559 bytes kernel obesity ;)

Yeah, true. It's really appalling. Make it a config option then, default
'n'. I will set it to 'y' for SGI_IP30. The Octane firmware has no
problems booting ELF64 (don't know about ELF32 though).

> a few kB for exeption vectors are mirrored to physical address 0.

They made a habit of this. The bastards. ;)

Stanislaw

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Apr 2004 10:18:54 +0200

>  Fortunately everything is not a PC. :-)

Actually the R10000 way to do something like this is to use the uncached
attribute like in the Origin.  It allows using the same physical address
space several times for different purposes.  So on node 0 of an Origin
indeed memory starts at physical address zero and there is no hole for
the firmware.  IP27 is afaik the only system using this R10000 feature
which surprises me a little due to the otherwise great similarity of these
two systems.

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Sat, 24 Apr 2004 10:23:34 +0200 (CEST)

> which surprises me a little due to the otherwise great similarity of these

 That precludes the firmware from being run cached, though.  Not very 
nice, especially for callbacks, but perhaps a bit easier to deal with.

---

From: Thiemo Seufer <[email-masked]>
Date: Sat, 24 Apr 2004 10:26:33 +0200

> problems booting ELF64 (don't know about ELF32 though).

The IP28 firmware refuses to boot ELF32.

Thiemo

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Apr 2004 10:26:55 +0200

> nice, especially for callbacks, but perhaps a bit easier to deal with.

Sane firmware copies itself to RAM at the earliest possible stage anyway -
ROMs are way too slow.

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Sat, 24 Apr 2004 10:49:09 +0200 (CEST)

> Sane firmware copies itself to RAM at the earliest possible stage anyway -

 Indeed, though it excludes the RAM used from the OS control (unless the 
OS wants to block itself from the access to callbacks).

 FYI, DEC copies only the bits it currently needs (and e.g. option ROMs 
typically cannot be directly executed at all as they often are 8-bit, but 
return word-aligned data) and when booting the OS certain callback vector 
entries point to RAM and others to ROM.