Subject: Old bug with 'gdb/dbxread.c' and screwed up MIPS symbolic debugging...

From: "Steven J. Hill" <[email-masked]>
Date: Wed, 31 Oct 2001 11:00:33 -0600

I have attempted to do as much research and testing as I possibly can before
posting this. I remember reading a thread associated with this problem in the
past, but things are still not working properly. I have also taken the liberty
of CC'ing the binutils and linux-mips list just in case.

GOAL
----
To use my KGDB stub with GDB and/or Insight to debug my embedded MIPS kernel
over the serial link utilizing symbolic debugging.

TOOLCHAIN
---------
binutils-2.11.92.0.10 (stock)
gcc-3.0.2 (stock)
glibc-2.2.3 (minor patches to ld-scripts and a small fixes for ipc/shm)
gdb-10312001 (fresh out of cvs this morning w/patch applied, see bottom)

CONFIGURATION LINES FOR TOOLS
-----------------------------
../binutils-2.11.92.0.10/configure --prefix=/opt/toolchains/mips
--target=mipsel-linux

AR=mipsel-linux-ar RANLIB=mipsel-linux-ranlib ../gcc-3.0.2/configure
--prefix=/opt/toolchains/mips --target=mipsel-linux i686-pc-linux-gnu
--with-newlib --disable-shared --enable-languages=c

BUILD_CC=gcc CC=mipsel-linux-gcc AR=mipsel-linux-ar AS=mipsel-linux-as
RANLIB=mipsel-linux-ranlib ../glibc-2.2.3/configure
--prefix=/opt/toolchains/mips/mipsel-linux mipsel-linux
--build=i686-pc-linux-gnu --enable-add-ons --with-elf --disable-profile
--with-headers=/opt/toolchains/mips/mipsel-linux/include
--mandir=/opt/toolchains/mips/man --infodir=/opt/toolchains/mips/info

AR=mipsel-linux-ar RANLIB=mipsel-linux-ranlib ../gcc-3.0.2/configure
--prefix=/opt/toolchains/mips --target=mipsel-linux i686-pc-linux-gnu
--with-gxx-include-dir=/opt/toolchains/mips/mipsel-linux/include
--mandir=/opt/toolchains/mips/man --infodir=/opt/toolchains/mips/info
--enable-languages=c,c++ --enable-threads

../gdb-10312001/configure --prefix=/opt/toolchains/mips --target=mips-linux-elf

KERNEL
------
Last 2.4.5 release from oss.sgi.com CVS

TYPICAl KERNEL COMPILE LINE
---------------------------
mipsel-linux-gcc -I /opt/mips/settop/include/asm/gcc -D__KERNEL__
-I/opt/mips/settop/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
-fno-strict-aliasing -g -G 0 -mno-abicalls -fno-pic -mcpu=r5000 -mips2
-Wa,--trap -pipe  -I .. -I /opt/mips/settop/fs  -funsigned-char   -c -o
xfs_griostubs.o xfs_griostubs.c
Assembler messages:
Warning: The -mcpu option is deprecated.  Please use -march and -mtune instead.
Warning: The -march option is incompatible to -mipsN and therefore ignored.

PROBLEM REPORT
--------------
I am using a NEC MIPS VR5432 in little endian and 32-bit mode. If I run
'mipsel-linux-objdump -d vmlinux', I get addresses starting with 0x8000XXXX.
With older toolchains I remember getting 0xffffffff8000XXXX. My kernel boots
fine and patiently waits for GDB to connect. If I use GDB stock from CVS
without applying any patches, the following output occurs:

This GDB was configured as "--host=i686-pc-linux-gnu --target=mips-linux-elf"...
(gdb) target remote /dev/ttyS1
Remote debugging using /dev/ttyS1
0x80012c88 in breakinst () at 1879
1879            sock_unregister(PF_PACKET);
(gdb) bt
#0  0x80012c88 in breakinst () at af_packet.c:1879
#1  0x8020aabc in brcm_irq_setup () at irq.c:421
#2  0x8020aaf0 in init_IRQ () at irq.c:434
#3  0x801fc83c in start_kernel () at init/main.c:524
#4  0x801fd6f8 in init_arch (argc=160, argv=0xb3000000, envp=0x2, 
    prom_vec=0xbf) at setup.c:425
(gdb) c
Continuing.

Program received signal SIGTRAP, Trace/breakpoint trap.
0x80012c88 in breakinst () at af_packet.c:1879
1879            sock_unregister(PF_PACKET);
(gdb) bt
#0  0x80012c88 in breakinst () at af_packet.c:1879
#1  0x8001a554 in sys_create_module (name_user=0x10001dc8 "brcmdrv", 
    size=713264) at module.c:305
(gdb) c
Continuing.

Which is clearly wrong. 'breakinst' is clearly not in that file, but all the
other symbolics in the backtrace are correct. Then if I go to insert a module,
'breakinst' again is decoded at the wrong place, but it gets 'sys_create_module'
module symbol decoded right. I will point out that 'breakinst' is defined in
'arch/mips/kernel/gdb-stub.c' and FWIW, looks like:

        __asm__ __volatile__("
                        .globl  breakinst
                        .set    noreorder
                        nop
breakinst:              break
                        nop
                        .set    reorder
        ");

"SOLUTION"
----------
On August 15, H.J. Lu applied a patch to 'gdb/dbxread.c' shown here:

   diff -urN -x CVS work/gdb/dbxread.c gdb-5.0-08162001/gdb/dbxread.c
   --- work/gdb/dbxread.c  Tue Oct 30 16:33:33 2001
   +++ gdb-5.0-08162001/gdb/dbxread.c      Wed Aug 15 00:02:28 2001
   @@ -951,7 +951,10 @@
        (intern).n_type = bfd_h_get_8 (abfd, (extern)->e_type);            \
        (intern).n_strx = bfd_h_get_32 (abfd, (extern)->e_strx);           \
        (intern).n_desc = bfd_h_get_16 (abfd, (extern)->e_desc);           \
   -    (intern).n_value = bfd_h_get_32 (abfd, (extern)->e_value);         \
   +    if (bfd_get_sign_extend_vma
(abfd))                                       \
   +      (intern).n_value = bfd_h_get_signed_32 (abfd,
(extern)->e_value);       \
   +    else                                                               \
   +      (intern).n_value = bfd_h_get_32 (abfd, (extern)->e_value);       \
      }
 
    /* Invariant: The symbol pointed to by symbuf_idx is the first one

If I back out this change, my debug output is "correct", but I no longer
have the nice line numbers and files decoded for me:

(gdb) target remote /dev/ttyS1
Remote debugging using /dev/ttyS1
0x80012c88 in breakinst ()
(gdb) bt
#0  0x80012c88 in breakinst ()
#1  0x8020aabc in brcm_irq_setup ()
#2  0x8020aaf0 in init_IRQ ()
#3  0x801fc83c in start_kernel ()
#4  0x801fd6f8 in init_arch ()
(gdb) c
Continuing.

Also, if I attempt to back out this patch from the latest insight CVS code,
it has not affect. Insight would still decode 'breakinst' at 'af_packet.c'.

CONCLUSION
----------
Conclusion? Uhh, things don't work. I greatly appreciate input from people
on this issue and hope I have supplied enough detail. I don't want to start
digging into the gdb source too deep without hearing other people's opinions.
Thanks.

-Steve

-- 
 Steven J. Hill - Embedded SW Engineer

---

From: Daniel Jacobowitz <[email-masked]>
Date: Wed, 31 Oct 2001 11:32:08 -0500

> PROBLEM REPORT

Well, the change in objdump output is purely cosmetic.  For 32bit
object formats we just truncate the display now.

> 'arch/mips/kernel/gdb-stub.c' and FWIW, looks like:

Does this happen for any other symbol than breakinst?  Breakinst is
effectively a function with no debugging info.  That case historically
causes us problems, so we probably missed another need for sign
extension.

> "SOLUTION"

If you back it out, I believe we just give up in confusion :)  This is
int the reading of stabs info.  breakinst has no stabs info, so it's
getting its line info somewhere else.

Please point me at a copy of your kernel binary with debug info, and
I'll look into it.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

---

From: Andrew Cagney <[email-masked]>
Date: Wed, 31 Oct 2001 13:11:25 -0500

> object formats we just truncate the display now.

As an aside, is there an option to turn this truncation off?  The vr5432 
as and ld will still pass around 64 bit addresses.

> If you back it out, I believe we just give up in confusion  [:)] This is

It might - assembler debugging ...

> Please point me at a copy of your kernel binary with debug info, and

Yes, you want to look for a version of breakinst that isn't sign 
extended.  Since pulling the above patch helped it won't be in .stabs so 
the symbol table?

Andrew

---

From: Daniel Jacobowitz <[email-masked]>
Date: Wed, 31 Oct 2001 17:47:49 -0500

> as and ld will still pass around 64 bit addresses.

It shouldn't happen in that case, I think.  The vr5432 is configured as
a mips64 target, isn't it?

> It might - assembler debugging ...

I don't think it does, at least...

> extended.  Since pulling the above patch helped it won't be in .stabs so 

It's not that breakinst isn't sign extended.  This is very much like
the address ranges issue that came up with -ffunction-sections or
linkonce sections.

I'm writing this as I debug.  Excuse the flow of consciousness (or skip
down to the end!).

Here's our bug:
(top-gdb) p/x *b
$21 = {startaddr = 0x34, endaddr = 0xffffffff80215314, function = 0x0,
  superblock = 0x0, gcc_compile_flag = 0x2, nsyms = 0x6,
  sym = {0x8755bbc}}

The startaddr is obviously wrong.  This is the first symtab listed for
kernel.  So where does that startaddr come from?

(By the way, our debugging of long longs is abysmal.  I already filed a
PR about this I think.  It makes tracking 64bit CORE_ADDRs a real
pain; they're printed with the upper half garbage.)

The answer is that the startaddr comes from the psymtab.  During psymbol
reading:

Hardware watchpoint 24: *$139

Old value = 18446744069414584372
New value = 52

at:

630                         && (CUR_SYMBOL_VALUE
631                             != ANOFFSET (objfile->section_offsets,
632                                          SECT_OFF_TEXT (objfile))))))
633               {
634                 TEXTLOW (pst) = CUR_SYMBOL_VALUE;
635                 textlow_not_set = 0;
636               }
637     #endif /* DBXREAD_ONLY */
638             add_psymbol_to_list (namestring, p - namestring,
639                                  VAR_NAMESPACE, LOC_BLOCK,

Here's the offending stabs entry:
317176 FUN    0      1870   0000000000000034 1689303 packet_exit:f(0,20)

i.e. it has value 0x34 (52).  That's bad.

Now, there's two things wrong here.  One of them is the bad value.  I
think that I've already seen this problem, and that it has something to
do with the way stabs are and used to be emitted.  packet_exit is
presumably in the .text.exit section, which is presumably the problem. 
Before linking, the stab looked like:

2971   FUN    0      1870   0000000000000000 159366 packet_exit:f(0,20)

and had a relocation:
0000000000008b58 R_MIPS_32         .text.exit
unless I miss my guess.

So it looks like binutils did not relocate the stabs for .text.exit properly.

Why?  It's pretty simple; there was nothing to relocate it to.  From the
kernel's linker script:

  /* Sections to be discarded */
  /DISCARD/ :
  {
        *(.text.exit)
        *(.data.exit)
        *(.exitcall.exit)
  }

So instead of the subtle error we get in objfiles containing multiple
sections, which we'll still need to deal with for the kernel for
.text.init, we have a completely bogus result.

We need to discard the stabs records for discarded symbols.  Of course,
we're just reading the psymtab in when we get here.  We don't have
symbols yet.  We could do this by a second pass after reading, instead
of the hack with textlow, but that's gross.

This makes it impossible to debug at least MIPS kernels with stabs and
gdb, so I very much want to fix it.  I'm not sure how this works on
x86, but I'd guess it had to do with differences in relocation types. 
Anyone have an example handy?

Meanwhile, Steven, as a quick hack - try removing the /DISCARD/ bit
from your linker script and relinking.  What happens?  Does everything
else seem to work?