Subject: ptrace and floating point related kernel crash

From: Daniel Jacobowitz <[email-masked]>
Date: Sat, 2 Apr 2005 17:23:53 -0500

Here's a nasty little bug I encountered while debugging some related
problems in GDB.

Compile and run the attached program; I'm not sure if it will demonstrate
the problem on anything with hardware FPU, but at least it works on an SB-1
(using a 32-bit kernel).  The program itself runs fine.  Debug it with GDB,
and set a breakpoint on the ctc1 instruction.  Before it executes, print out
$fsr; it will probably be 0.  After trying to copy 0xf0102 into FSR, print
$fsr again; it will be 0x102.  The program will still complete OK.

Now try again.  After the ctc1 instruction, tell gdb "set $fsr = 0xf0102".
Then continue; the kernel locks up before the program is done.

The extra bits are two bits in the cause field, and two bits in the
reserved-write-as-zero field.  I'm not sure whether setting the reserved
bits is to blame, or whether setting the cause bits raises a floating point
exception in the kernel during context switching.  In any case, it looks
like we ought to be masking out some bits before saving the fcr31 value in
ptrace.

-- 
Daniel Jacobowitz
CodeSourcery, LLC