Subject: User applications

From: Carsten Langgaard <[email-masked]>
Date: Mon, 08 Jan 2001 10:40:12 +0100

I have a few questions about user applications.

When a new user process is started will its user space be cleared by the
kernel or is there a potential leak from an older user process ?
What about the registers values, are they cleared for each new user
application or will it simply contain the current value it got when the
user application is started ?
How can you flush the data and instruction cashes from a user
application ?

/Carsten

--
_    _ ____  ___   Carsten Langgaard   Mailto:[email-masked]
|\  /|||___)(___   MIPS Denmark        Direct: +45 4486 5527
| \/ |||    ____)  Lautrupvang 4B      Switch: +45 4486 5555
  TECHNOLOGIES     2750 Ballerup       Fax...: +45 4486 5556
                   Denmark             http://www.mips.com

---

From: Michael Shmulevich <[email-masked]>
Date: Mon, 08 Jan 2001 15:30:35 +0200

> kernel or is there a potential leak from an older user process ?

Usually it is defied by the loader. If the data section contents is set to
LOAD, then the contents of the section will be loaded from disk (no leak),
if not -- whatever values left i nmemory will be there, or exactly, the
virtual page of some other proccess that was swapped out or ended.

> user application is started ?

It depends on the context switch algorithm of the processor, I think.

> How can you flush the data and instruction cashes from a user

As far as I understand, ASID must take care of it. It contains unique IDs
per process virtual space, so that even
though virtual addresses may be found in TLB, their ASID will be different,
causing TLB miss and probably page fault.

> /Carsten

Michael.

---

From: Carsten Langgaard <[email-masked]>
Date: Mon, 08 Jan 2001 14:52:10 +0100

> causing TLB miss and probably page fault.

My problem is that I want to make self-modified code, so I need to flush both
the instruction and data cache.

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Mon, 8 Jan 2001 15:14:38 +0100

> virtual page of some other proccess that was swapped out or ended.

Note that what you are describing here is the "exec()" behavior.
I believe Carsten was talking about what happens on a "fork()".

> It depends on the context switch algorithm of the processor, I think.

On a fork() (or presumably clone()) operation, the set of registers
is copied.  Loading a new program ("exec()") should set up the
registers that point to the base of the new stack, the environment,
etc.  Historically, it's up to the runtime startup code ("crt0" in old
Unix systems) to do any other register initialization.

> causing TLB miss and probably page fault.

That won't necessarily affect the caches, though.  While it
would be possible to do so, I don't believe any existing
MIPS implementations include ASID in the cache tags.
Hits are determined by an address match, period.

Back in the Ancient Old Days of System V, every architecture
had an architecture-specific system call entry, the first parameter
of which expressed what needed to be done.  Do we have
such a thing in Linux?   That would be the logical place to
things like cache flush and the atomic operations that were
being discussed here a couple of weeks ago.

            Regards,

            Kevin K.

---

From: Carsten Langgaard <[email-masked]>
Date: Mon, 08 Jan 2001 15:16:16 +0100

> being discussed here a couple of weeks ago.

I think I just found it.
The system call is sysmips(FLUSH_CACHE).

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 15:16:46 +0100 (MET)

> virtual page of some other proccess that was swapped out or ended.

 What!???  I'm assume you are writing about executing a new program and
not forking a new process here.  In the latter case no memory is changed. 
When you exec a new program, any allocated memory is cleared by the kernel
before returning to the user space.  It would be a huge security hole
otherwise.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 16:07:31 +0100 (MET)

> being discussed here a couple of weeks ago.

 The only case caches need to be synchronized is modifying some code.  The
ptrace syscall does it automatically for text writes -- it's needed and
used by gdb to set breakpoints, for example.  For other code there is
cacheflush() which allows you to flush a cache range relevant to a given
virtual address (I see it's not implemented very well at the moment).

 Obviously, you don't want to allow unprivileged users to flush caches as
a whole as it could lead to a DoS.

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Mon, 8 Jan 2001 16:21:18 +0100

----- Original Message -----
From: "Maciej W. Rozycki" <[email-masked]>
To: "Kevin D. Kissell" <[email-masked]>
Cc: <[email-masked]>; "Carsten Langgaard" <[email-masked]>;
"Michael Shmulevich" <[email-masked]>
Sent: Monday, January 08, 2001 4:07 PM
Subject: Re: User applications

>  The only case caches need to be synchronized is modifying some code.

Which just happens to be the case under discussion here.

>The  ptrace syscall does it automatically for text writes -- it's needed
and
> a whole as it could lead to a DoS.

By that logic, we should not allow users to allocate more virtual
memory than there is physical memory in the system!  A pathological
swap program is arguably far a far worse denial of service attack
than flushing the caches - so long as by "flush" we mean invalidate
with writeback (on copyback caches), of course.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 16:40:06 +0100 (MET)

> swap program is arguably far a far worse denial of service attack

 There are limits -- see `info setrlimit'.  There is no way to prevent a
program from executing:

while (1) flush_cache_all();

though but the system's performance would suffer much.  Remember there is
real world out there... 

 Which means sysmips(FLUSH_CACHE, ...) needs to be fixed or removed. 

> with writeback (on copyback caches), of course.

 What's wrong with cacheflush(addr, count, which) that actually checks if
<addr; addr+count> lies within the caller's address space before
performing the flush and returns -EPERM otherwise?  It would make the
caller crawl like a turtle if it wished to but it would leave other
processes alone.

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 8 Jan 2001 14:03:04 -0200

> The system call is sysmips(FLUSH_CACHE).

Don't.  Sysmips(FLUSH_CACHE, ...) only allows very coarse flush operation,
that is flushing all caches.  The whole sysmips(2) call exists in Linux
only as a stone age compatibility thing.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 8 Jan 2001 14:05:06 -0200

> a whole as it could lead to a DoS. 

You obviously want to allow partial cache flushes or trampolines don't work
and flushing the entire cache can be constructed from that.

  Ralf

---

From: Carsten Langgaard <[email-masked]>
Date: Mon, 08 Jan 2001 17:23:20 +0100

> and flushing the entire cache can be constructed from that.

What we need is a mechanism to partial invalidate the I-cache and a mechanism
to write-back and/or invalidate the D-cache.

/Carsten

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 8 Jan 2001 14:25:13 -0200

> kernel or is there a potential leak from an older user process ?

A new process is started by the clone(2) or fork(2) syscalls.  Module the
options that can be passed to clone(2) the two only create an identical copy
of the invoking process, so they're designed to leak information by design ;-)

execve(2) replaces the existing mappings with a new process image loaded
from files plus a newly created stack area.  No old mappings survive, so
there in memory there is no information leak.

> user application is started ?

We make no attempt at the integer registers for a new process, so some
information might be leaked in registers.  All the callee saved registers
will be passed unchanged to the child process; the caller saved registers
except those that are used as syscall return values will return random
garbage.  Floating point registers will be cleared with SNANs as soon
as the process is attempting to use a FPU for the first time, that is
we won't leak information via fpu registers.

(Ooops, we're not Orange Book B1 compliant, how sad ;-)

> How can you flush the data and instruction cashes from a user application ?

cacheflush(2).  See man page.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 8 Jan 2001 14:27:29 -0200

> caller crawl like a turtle if it wished to but it would leave other

cacheflush(2) actually is supposed to handle things that way.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 8 Jan 2001 14:30:14 -0200

> to write-back and/or invalidate the D-cache.

There is this nice little man page which should even be installed on your
Linux/Inhell box:

CACHEFLUSH(2)       Linux Programmer's Manual       CACHEFLUSH(2)

NAME
       cacheflush  -  flush  contents  of instruction and/or data
       cache

SYNOPSIS
       #include <asm/cachectl.h>

       int cacheflush(char *addr, int nbytes, int cache);

DESCRIPTION
       cacheflush flushes contents of indicated cache(s) for user
       addresses  in the range addr to (addr+nbytes-1). Cache may
       be one of:

       ICACHE Flush the instruction cache.

       DCACHE Write back to memory and  invalidate  the  affected
              valid cache lines.

       BCACHE Same as (ICACHE|DCACHE).

RETURN VALUE
       cacheflush  returns 0 on success or -1 on error. If errors
       are detected, errno will indicate the error.

ERRORS
       EINVAL cache parameter is not one of  ICACHE,  DCACHE,  or
              BCACHE.

       EFAULT Some   or   all   of  the  address  range  addr  to
              (addr+nbytes-1) is not accessible.

BUGS
       The current implementation ignores  the  addr  and  nbytes
       parameters.   Therefore always the whole cache is flushed.

NOTE
       This system call is only available on MIPS based  systems.
       It should not be used in programs intended to be portable.

Linux 2.0.32                27 June 95                          1

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 17:34:05 +0100 (MET)

> and flushing the entire cache can be constructed from that.

 It depends on the implementation.  The current one obviously allows it. 
;-) 

 I'm writing of the design.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 17:43:24 +0100 (MET)

> cacheflush(2) actually is supposed to handle things that way.

 Didn't I write it clearly-enough? ;-)

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 8 Jan 2001 14:41:11 -0200

>  Didn't I write it clearly-enough? ;-) 

return -EOVERBLOODINCAFFEIN;

  Ralf

---

From: Carsten Langgaard <[email-masked]>
Date: Mon, 08 Jan 2001 17:50:51 +0100

Exactly that I need, but I don't think it is implemented properly for mips.
It simply flushes all the caches, no matter what options you gives it.

/Carsten

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 17:40:23 +0100 (MET)

> to write-back and/or invalidate the D-cache.

 What's wrong with the already mentioned cachectl?

$ mipsel-linux-objdump -T /usr/mipsel-linux/lib/libc-2.2.so | grep cachectl
00000000600ca0a0  w   DF *ABS*  0000000000000000  GLIBC_2.0   cachectl
$ ls /usr/mipsel-linux/include/sys/cachectl.h
/usr/mipsel-linux/include/sys/cachectl.h

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 8 Jan 2001 15:42:32 -0200

> /usr/mipsel-linux/include/sys/cachectl.h

cachectl(2) is a syscall that is manipulates the cachability of a memory
area.  And not yet implemented ...

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 18:58:57 +0100 (MET)

> area.  And not yet implemented ...

 s/cachectl$/cacheflush/, of course (but the header is still valid).

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 8 Jan 2001 18:56:55 +0100 (MET)

> It simply flushes all the caches, no matter what options you gives it.

 Yep, it's just a limitation of the implementation.  But it's the right
function and now that I know of the problem it goes to my to do list (but
processing of the list is not that fast these days, so anyone feel free to
do it oneself).

---

From: Michael Shmulevich <[email-masked]>
Date: Tue, 09 Jan 2001 13:49:29 +0200

As a side question, I would like to to know why exactly the CPU cache operations
are
promoted to the syscall status? What is the situation that a user in its program
would like
to call cacheflush() ? Unless, of course, he is doing DoS.

I can understand why we need this in kernel, for context switch, for example, but
as a syscall?...

Michael.

---

From: Alan Cox <[email-masked]>
Date: Tue, 9 Jan 2001 12:17:26 +0000 (GMT)

> to call cacheflush() ? Unless, of course, he is doing DoS.

A cache flush is not a denial of service attack. Its no less effective than a
1Mb memcpy 

> I can understand why we need this in kernel, for context switch, for example, but

Self modifying code, dynamic compilation, glibc trampolines

---

From: Geert Uytterhoeven <[email-masked]>
Date: Tue, 9 Jan 2001 13:15:47 +0100 (MET)

> I can understand why we need this in kernel, for context switch, for example, but

For trampolines. These are small pieces of code created on the stack, and
require flushing of the caches before they are excuted.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven ------------- Sony Software Development Center Europe (SDCE)
[email-masked] ------------------- Sint-Stevens-Woluwestraat 55
Voice +32-2-7248626 Fax +32-2-7262686 ---------------- B-1130 Brussels, Belgium

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Tue, 9 Jan 2001 14:00:01 +0100 (MET)

> to call cacheflush() ? Unless, of course, he is doing DoS.

 Any software that modifes text needs it.  For example the dynamic linker
or libdl.