Subject: Prefetches in memcpy

From: Carsten Langgaard <[email-masked]>
Date: Tue, 05 Nov 2002 14:45:47 +0100

I have reported this before and it also gave a lot of responses, but
nothing has been done about, unfortunately :-(

The problem is the prefetches in the memcpy function in the kernel.
There is spread a number of PREF instructions in the memcpy function,
but there is no check if we are prefetching out-side the areas we are
copying to/from. This is extremely dangerous because we might prefetch
out-side the physical memory area, causing e.g. a bus error or something
even more nasty.

I recently found something even nastier, it could also hit a DMA buffer
region, and thereby break the PCI DMA flushing scheme.
For example if the kernel is doing a memcpy from an area that's next to
a DMA buffer area, we could end up in a situation where, we are
prefetching
data into the cache from a memory location that is used for DMA transfer
and owned by the device, but the DMA transfer has not yet completed.
We then end up in a situation, where the memory and cache is out of sync
and the cache is containing some old data.

So we definitely need to do something about the prefetches in the memcpy
function.
We can either get rid of all the prefetches or make sure we don't
prefetch out side the "memcpy" area.

/Carsten

--
_    _ ____  ___   Carsten Langgaard   Mailto:[email-masked]
|\  /|||___)(___   MIPS Denmark        Direct: +45 4486 5527
| \/ |||    ____)  Lautrupvang 4B      Switch: +45 4486 5555
  TECHNOLOGIES     2750 Ballerup       Fax...: +45 4486 5556
                   Denmark             http://www.mips.com

---

From: Ralf Baechle <[email-masked]>
Date: Tue, 5 Nov 2002 16:38:07 +0100

On Tue, Nov 05, 2002 at 02:45:47PM +0100, Carsten Langgaard wrote:

> function.  We can either get rid of all the prefetches or make sure we
> don't prefetch out side the "memcpy" area.

We could fix the prefetch into DMA buffer problem with an extra flush but
that's going to be expensive, I rather think we should avoid prefetches.
As Kevin explained KSEG1 is a loophole in the spec so we can't really say
what the behaviour of memcpy will be in KSEG1.

So I think the fix will have to be:

 - Avoid prefetching beyond the end of the copy area in memcpy and memmove.
 - Introduce a second variant of memcpy that never does prefetching.  This
   one will be safe to use in KSEG1 / uncached XKPHYS also and will be used
   for memcpy_fromio, memcpy_toio and friends.

  Ralf

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Tue, 5 Nov 2002 17:13:48 +0100

From: "Ralf Baechle" <[email-masked]>
>    for memcpy_fromio, memcpy_toio and friends.

Assuming we had a version that prefetched exactly to the end
of the source memory block and no further, why would we need
the second variant?

            Kevin K.

---

From: Ralf Baechle <[email-masked]>
Date: Tue, 5 Nov 2002 17:29:35 +0100

> of the source memory block and no further, why would we need

Because the source of memcpy_fromio and the destination of memcpy_toio are
some I/O address, typically something like a shared memory region on a
network card, which is accessed uncached.  The uncached address region
might be mapped in KSEG2/KSEG3 or accessed through an uncached region of
XKPHYS or KSEG1 where as I recall your statment the effect of prefetch
instructions is undefined.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Tue, 5 Nov 2002 17:41:02 +0100

> of the source memory block and no further, why would we need

Uh...  Maybe missread your mail the first time.  We're also prefechting
the destination area, of course with a different hint.  The question is
what the hints for destination are actually doing, the specs are fairly
vague on that and seem to leave the decission to the implementor.  Do
we really want to try ...

  Ralf

---

From: Alan Cox <[email-masked]>
Date: 05 Nov 2002 17:26:11 +0000

> of the source memory block and no further, why would we need

memcpy_to/from_io needs to do strictly ordered fetches on some devices
which fake a memory range with a fifo for performance (pci bursting
dirty tricks)