Subject: mips 32 bit HIGHMEM support

From: <[email-masked]>
Date: Wed, 8 Oct 2003 16:00:33 -0600 (MDT)

Hello Ralf,

I work for HP in Boise, ID, USA.  For the last six months I have been
working with Linux in HP Laser jet Printers.  It is running in a formatter
board that features an ASIC with a MIPS 20Kc core.

The original Linux port was achieved by others working before me.  I began
working on applications for it when I started here (to make it print
actually).  As I have tried to milk more performance out of this board the
recurrent bottleneck has been the fact that the kernel only recognizes
512Mb of RAM.  So my current endeavor is fix the kernel so it sees the 1G
of RAM that is physically present in the system.

I reconfigured and rebuilt the kernel with HIGHMEM support.  HIGHMEM is
seen and booting up goes splendidly until init (pid 1) kicks off.  Early
on in the execution of init the system crashes.  Specifically it crashes
in do_page_fault().  A couple of strange things I have noticed:

1.  All memory pages are divied up between the DMA Zone and the HIGHMEM
Zone at bootup.  the NORMAL zone gets 0 pages.  Not sure if this is
normal.

2.  The virtual address that do_page_fault() chokes on is 0x00000000. 
This doesn't seem like a reasonable address for init to be accessing
especially considering its mem map only contains the chunks
0x10000000-0x10001000 and 0x400000-0x40b000 (I got these by traversing
init's mm->mm_rb tree).

I decided to contact you about this after digging up this old posting of
yours:

>Is anybody using 32-bit MIPS CPUs which have more than 512mb of memory or
>to be more exact have RAM that isn't accessible via the KSEG0 / KSEG1
>window?
>
>So far I haven't ever seen such a machine.  For 64-bit CPUs the right
>thing to do is easy - use a 64-bit kernel.  But for 32-bit CPUs the Intel
>highmem stuff in the memory managment now gives us a sane way to use
>the memory of such configuration with just a little bit of extra code.
>
>So if anybody wants support for such a configuration, please drop me a
>note.
>
>Thanks,
>
>  Ralf

I think this describes my machine.

If this is an issue who's solution is old news please point me to the
solution.  In any case any ideas or guidance regarding this crash would be
greatly appreciated.

Regards, Andrew Stone

---

From: Ralf Baechle <[email-masked]>
Date: Thu, 9 Oct 2003 16:03:20 +0200

> board that features an ASIC with a MIPS 20Kc core.

Reminds me of the HP Laserjet code in the kernel.  Anybody still using
or testing that?

> I decided to contact you about this after digging up this old posting of

[... ancient posting deleted ...]

> solution.  In any case any ideas or guidance regarding this crash would be

In the meantime there is a highmem implementation for the MIPS kernel.
It's got a limitation - it only works on physically indexed D-caches or
more exactly processors that don't suffer from cache aliases.  On
processors that have such aliases the necessary flushes are rather bad
for performance so this currently simply isn't suported.

Anyway, for a 64-bit processor such as the 20Kc I suggest a 64-bit kernel.
Highmem is a pain and 64-bit is the cure.

  Ralf

---

From: "Steve Scott" <[email-masked]>
Date: Thu, 9 Oct 2003 15:11:36 -0700

I ran across what looks like a bug in the 'kernel_entry' function in
linux/arch/mips/kernel/head.S while chasing another problem. Our
version of kernel_entry is for 2.4.17. 2.4.22 seems to have
the same problem.

kernel_entry initializes the kernel stack pointer 'kernelsp'. But then
immediately after this clears the bss, which has the side effect of setting
kernelsp to 0. In our system, on initial entry to cpu_idle(), kernelsp is 0.

The bug "heals" itself the first time schedule() is called. But, if for some
reason CP0_STATUS doesn't have CU0 set at startup (which would be
bad for other reasons), and you get an exception before the first call to
schedule() (e.g., the syscall to create "init"), the exception handler will
try to save registers starting from kernelsp, which is 0.

from head.S:

    NESTED(kernel_entry, 16, sp)
    .
    .
    .
    /*
     * Stack for kernel and init, current variable
     */
    la $28, init_task_union
    addiu t0, $28, KERNEL_STACK_SIZE-32
    subu sp, t0, 4*SZREG

    sw t0, kernelsp                /* <-- this is going to get overwritten below when bss
is cleared... [srs] */

    /* The firmware/bootloader passes argc/argp/envp
     * to us as arguments.  But clear bss first because
     * the romvec and other important info is stored there
     * by prom_init().
     */
    la t0, _edata                    /* <-- here is the code that eventually clears
kernelsp [srs] */
    sw zero, (t0)
    la t1, (_end - 4)
1:
    addiu t0, 4
    bne t0, t1, 1b
    sw zero, (t0)

   jal init_arch
    nop
  END(kernel_entry)

--steve

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 10 Oct 2003 15:23:12 +0200

> kernelsp to 0. In our system, on initial entry to cpu_idle(), kernelsp is 0.

Funny this wasn't noticed for so long.  Fix is in CVS, thanks for reporting.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 10 Oct 2003 16:59:49 +0200

> for performance so this currently simply isn't suported.

Small update - I changed the memory managment code to ignore highmem
on cache configuration that are not supported by the highmem code.

  Ralf

---

From: <[email-masked]>
Date: Mon, 13 Oct 2003 17:15:19 -0600 (MDT)

Hi Ralf,

Thanks for the prompt response.

> Reminds me of the HP Laserjet code in the kernel.  Anybody still using

Yes, the code I am working with is basically a snapshot of the kernel
source with /arch/mips/hp-lj etc.

> Anyway, for a 64-bit processor such as the 20Kc I suggest a 64-bit
> kernel. Highmem is a pain and 64-bit is the cure.

To that end I have been trying to build a 64 bit kernel.  I think I am
going about it correctly.  Got ahold of

mips64el-linux-binutils-2.14-3.i386.rpm 
mips64el-linux-boot-gcc-2.95.4-9.i386.rpm

and got cranking.  Currently make is dying with the following output:

[root@l51dhcp252 ljlinux]# make ARCH=mips64 CROSS_COMPILE=mips64el-linux-
. scripts/mkversion > .tmpversion
mips64el-linux-gcc -D__KERNEL__ -I/home/astone/cf_burn_64/ljlinux/include
-Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing
-fno-common -I/usr/lib/gcc-lib/mips64el-linux/2.95.4/include
-fomit-frame-pointer -Dx_unity_x -I
/home/astone/cf_burn_64/ljlinux/include/asm/gcc -mabi=64 -G 0
-mno-abicalls -fno-pic -Wa,--trap -pipe  -DUTS_MACHINE='"mips64"'
-DKBUILD_BASENAME=version -c -o init/version.o init/version.c
mips64el-linux-gcc -D__KERNEL__ -I/home/astone/cf_burn_64/ljlinux/include
-Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing
-fno-common -I/usr/lib/gcc-lib/mips64el-linux/2.95.4/include
-fomit-frame-pointer -Dx_unity_x -I
/home/astone/cf_burn_64/ljlinux/include/asm/gcc -mabi=64 -G 0
-mno-abicalls -fno-pic -Wa,--trap -pipe   -DKBUILD_BASENAME=do_mounts -c
-o init/do_mounts.o init/do_mounts.c
cpp0: output pipe has been closed
mips64el-linux-gcc: Internal compiler error: program cc1 got fatal signal 11
make: *** [init/do_mounts.o] Error 1

Initial searches claim this is a bug in the compiler.  Could it be due to
something I am doing wrong?  Any guidance on building a 64 bit kernel
would be greatly appreciated.

Best Regards, Andrew Stone