Subject: [Q: Linux/SGI] IRIX executable memory map.

From: Miguel de Icaza <[email-masked]>
Date: Thu, 4 Sep 1997 21:45:31 -0500

Hello guys,

    Ok, it seems our irix_elfmap routine is just fine, I just found
out with a simple test case that the code is trying to access memory
from the location at 0x200000 which is making my IRIX executables
crash (this one is crashing inside usinit ()).

    Is there something magic about 0x20000 on IRIX that I should be
aware of? 

    This 0x20000 location is quite strange to me, since this is not a
region of memory that happens to be part of the executable according
to gdb (look at exhibit [1]).  So, if it does not come from here, the
obvious guess is that this memory region comes from mmaping something:
either /dev/zero, or from the sys_sgi (ELFMAP) system call, but
neither par nor strace show any trace of a virual mapping taking place
in this region (exhibit [2] shows this).

     The exact location of the crash is shown on my third dump [3].

Best wishes,
Miguel.

Exhibit [1]: Local exec file image, from gdb.
        Entry point: 0x400870
        0x00400134 - 0x00400147 is .interp
        0x004001c8 - 0x004001e0 is .reginfo
        0x004001e0 - 0x004002d8 is .dynamic
        0x004002d8 - 0x004002ec is .liblist
        0x0040050c - 0x0040070c is .dynsym
        0x004002ec - 0x00400401 is .dynstr
        0x00400404 - 0x0040050c is .hash
        0x0040072c - 0x0040082c is .msym
        0x0040082c - 0x00400870 is .MIPS.stubs
        0x00400870 - 0x00400a00 is .text
        0x00400a00 - 0x00400a20 is .init
        0x10001000 - 0x10001010 is .rodata
        0x10001010 - 0x10001048 is .got
        0x10001048 - 0x10001054 is .bss

Exhibit [2]: par output from running simple test program:
    2mS    usinit( 5349): execve(./usinit, 0x7fff2f64, 0x7fff2f6c) OK
    5mS    usinit( 5349): open(/lib/rld, O_RDONLY, 04) = 3
    5mS    usinit( 5349): read(3, <7f 45 4c 46 01 02 01 00 00 00 00>...,512) = 512 
    5mS    usinit( 5349): elfmap(3, 0x7fff2d58, 2) = 0xfb60000
    6mS    usinit( 5349): close(3) OK
    6mS    usinit( 5349): getpagesize() = 4096
    6mS    usinit( 5349): open(/dev/zero, O_RDONLY, 01757022324) = 3
    7mS    usinit( 5349): mmap(0xfbc4000, 16384, PROT_WRITE|PROT_READ,
			       MAP_PRIVATE, 3, 0) = 0xfbc4000
    7mS    usinit( 5349): close(3) OK
    9mS    usinit( 5349): syssgi(SGI_USE_FP_BCOPY, 0, 0x1dc,
				0xffffffff, 0x7fff2d70, 0xfb8eee8) = 0
    9mS    usinit( 5349): open(/dev/zero, O_RDONLY, 01757022324) = 3
    9mS    usinit( 5349): mmap(0xfbc8000, 32768, PROT_WRITE|PROT_READ,
				MAP_PRIVATE, 3, 0) = 0xfbc8000
    9mS    usinit( 5349): close(3) OK
    9mS    usinit( 5349): syssgi(SGI_TOSSTSAVE) OK
    9mS    usinit( 5349): write(2, "Test\n", 5) = 5
The IRIX program crashes here when running on Linux. 
   10mS    usinit( 5349): sysmp(MP_NPROCS) = 1
   10mS    usinit( 5349): open(tm, O_RDWR, 016) = 3
   10mS    usinit( 5349): fcntl(3, F_SETFD, 1) OK
   10mS    usinit( 5349): fcntl(3, F_SETLKW, type=F_WRLCK
				whence=SEEK_SET start=0 len=1) OK
   10mS    usinit( 5349): syssgi(SGI_USE_FP_BCOPY, 0x1000, 0x7fff1d5c,
				0xffffffff, 0x10001008, 0x7fff2fc0) = 0
   11mS    usinit( 5349): read(3, <de ad ba be 00 00 00 05 00 44 05 50
	00 44 05 a0>..., 4096) = 4096
[more system calls deleted]

Exhibit [3]:

The culprit piece of code, taken from the IRIX libc:

0xfa52a10 <_usinit+44>: lui $t6,0x20
0xfa52a14 <_usinit+48>: lw $t7,3584($t6)        <- This one is the culprit

This is loading something from 0x200e00

---

From: Miguel de Icaza <[email-masked]>
Date: Fri, 5 Sep 1997 13:42:08 -0500

IRIX unsolved misteries -- second update.

> crash (this one is crashing inside usinit ()).

I was debatting this with my bed sheets this morning and I believe
that this page may be a special trick for IRIX sproc()ed binaries.
Probably I should also turn off the cache on this page.

Miguel.

---

From: "William J. Earl" <[email-masked]>
Date: Fri, 5 Sep 1997 15:51:00 -0700

>     Is there something magic about 0x20000 on IRIX that I should be

     0x20000 (not 0x200000) is automatically mapped in a process
by the kernel when referenced.  See <sys/prctl.h> on IRIX for
the definitions.  Most of the fields you can probably ignore,
but you should set t_pid, t_rpid, and t_prid (where the latter is the value
of the $prid COP0 register).  Set t_pid and t_rpid to the pid of the process.
(They cannot really be different in IRIX.)

     The library is fetching the PID from t_pid instead of calling getpid()
for performance reasons.

     Simply automatically create the page in the page fault handler
when it is first referenced.  Treat as if it were an mmap() of /dev/zero
for 1 page.

---

From: "William J. Earl" <[email-masked]>
Date: Fri, 5 Sep 1997 15:58:07 -0700

> Probably I should also turn off the cache on this page.

     No, I think this page should be private to a given thread (sproc
or process), and it can be cached.

---

From: [email-masked] (Alan Cox)
Date: Sat, 6 Sep 1997 00:05:56 +0100 (BST)

> when it is first referenced.  Treat as if it were an mmap() of /dev/zero

Better to map it first. page fault is a far more critical path to stick
conditionals into. Even better if possible is to stick it into the userspace
dynamic loader for irix binaries. That however means not using the irix one
or having a "preloader" before it

Alan

---

From: Miguel de Icaza <[email-masked]>
Date: Fri, 5 Sep 1997 18:47:53 -0500

> the definitions.  

Ok, thanks for the pointer.  btw, the code seems to indicate that the
address is indeed 0x200000 (it does a lui $reg,0x20).  Oops.  Hold
on.  The sys/prctl agrees with me, it is 0x200000.

That structure looks very promising, I will go and fill it.

> Most of the fields you can probably ignore, but you should set
> t_pid, t_rpid, and t_prid (where the latter is the value of the
> $prid COP0 register).  Set t_pid and t_rpid to the pid of the
> process.  (They cannot really be different in IRIX.)

Ok.  There is another bit that looks promising: the t_cpu field, which
is supposed to have the cpu number where the process is executing.
Does IRIX update this on every context switch? 

> when it is first referenced.  Treat as if it were an mmap() of /dev/zero

Ok.  The code that I used last night to work around this was to map
a /dev/zero page at sys_irix_elfexec time, I think I will just poke
the right values there.

best wishes and thanks for this pointer!
Miguel.

---

From: Miguel de Icaza <[email-masked]>
Date: Fri, 5 Sep 1997 18:49:45 -0500

> or having a "preloader" before it

Nah.  We already have an IRIX specific ELF loader, so I will just
modify my existing mmap in that place.  No need for extra hacks if a
10 line change to irixelf.c fixes it.

Cheers,
Miguel.

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 6 Sep 1997 02:33:05 +0200 (MET DST)

Hi,

>      The library is fetching the PID from t_pid instead of calling getpid()

Byte Benchmarks ;-)

  Ralf

---

From: "William J. Earl" <[email-masked]>
Date: Fri, 5 Sep 1997 19:00:26 -0700

> on.  The sys/prctl agrees with me, it is 0x200000.

    Yes, you are correct.  

...
 > Does IRIX update this on every context switch? 

     Yes, on a multiprocessor, but it is a constant on a uniprocessor.

 > a /dev/zero page at sys_irix_elfexec time, I think I will just poke

      As someone else pointed out, mapping the page is sys_irix_elfexec
time is probably better, since it will avoid slowing down the page
fault handler.

---

From: "William J. Earl" <[email-masked]>
Date: Fri, 5 Sep 1997 19:03:01 -0700

> >      The library is fetching the PID from t_pid instead of calling getpid()

     Actually not, although we have thought about it.  getpid() itself is
a system call, even though it could just use the PRDA.  Note, however, that
IRIX processes which do not use the usema stuff generally do not have the
PRDA mapped, since we wanted to avoid allocating an extra page per process
for processes which do not use it.