Subject: NFS root with cache on

From: Quinn Jensen <[email-masked]>
Date: Thu, 01 Feb 2001 13:34:49 -0700

Is anyone else having trouble with NFS root on
the 2.4.0 kernel?  It won't come up with the
KSEG0 cache on unless I pepper the network driver
with flush calls.

I'm using the new mips32 mm and cache code with
my own patches to deal with the IDT 334's weird
way bit (it's clear up at bit 12 even though the
cache is only 2k).

Quinn

---

From: Ralf Baechle <[email-masked]>
Date: Sun, 4 Feb 2001 19:44:52 -0800

> KSEG0 cache on unless I pepper the network driver

That's expected for most old network drivers that don't yet use the
new PCI DMA API documented in Documentation/DMA-mapping.txt.

What driver is this?

  Ralf

---

From: Quinn Jensen <[email-masked]>
Date: Mon, 05 Feb 2001 09:50:51 -0700

> new PCI DMA API documented in Documentation/DMA-mapping.txt.

Both the stock 2.4.0 tulip and eepro100 drivers.  The
problem doesn't happen when I go back to 2.3.99pre8.

In the tulip driver, both the rx and tx ring descriptors
are adjusted with pci_alloc_consistent() and are only
touched through KSEG1.

But I must be totally clueless because in both kernels
tulip_rx() calls pci_unmap_single() which does nothing.
But the skb data pointers all point to KSEG0.  With
cache on, how in the world will the kernel be able to
see what just got DMA'd into the skb?

Another thing that has been haunting me is that
in 2.3.99pre8, kmalloc() has and #ifdef __mips__ that
flushes the cache and bumps the address up to KSEG1.
This is gone in 2.4.0, but from what I can tell, this
case didn't happen for skb allocations (i.e. dev_alloc_skb)
because they only set GFP_ATOMIC, and not GFP_DMA.

Quinn

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Mon, 5 Feb 2001 17:55:39 +0100

> flushes the cache and bumps the address up to KSEG1.

That's really hideous!  When did that happen?
It'll sure help cache coherency problems, but
the performance impact must have been monstrous.

            Kevin K.

---

From: Jun Sun <[email-masked]>
Date: Mon, 05 Feb 2001 10:07:18 -0800

> problem doesn't happen when I go back to 2.3.99pre8.

Quinn,

Did you set rx_copybreak to 1518?  I sent patches long time ago to the driver
authors for MIPS, but I am not sure they are not there.

Jun

---

From: Quinn Jensen <[email-masked]>
Date: Mon, 05 Feb 2001 11:42:57 -0700

[email-masked] wrote:

>>> That's expected for most old network drivers that don't yet use tye
he
> authors for MIPS, but I am not sure they are not there.

Jun,

I have tried that in this case but it didn't help,
because the receive skb data pointers all point to
the KSEG0 view of the data anyway.  But the maddening
thing is, when I force them to KSEG1 I still see the
NFS timeout problem.  So I'm starting to wonder if
the problem is higher up in the stack.  But I do know
that the behavior does not occur on a similarly configured
x86 build of the same kernel.

Quinn

---

From: Jun Sun <[email-masked]>
Date: Mon, 05 Feb 2001 11:15:19 -0800

Quinn Jensen wrote:
> the KSEG0 view of the data anyway. 

I looked into similar problems a while back.  If I remeber correctly, the data
pointers do point to kseg0.  It is up to the driver to do appropriate
dma_cache_invalidate() (or some functions to that effect) at certain places.

What is the CPU?  It seems logical to suspect about the dma cache routines.

Jun

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 5 Feb 2001 11:50:26 -0800

> authors for MIPS, but I am not sure they are not there.

Copybreak is just an optimization.  So even with this unused or set to a
wrong value the driver should work.

  Ralf

---

From: Quinn Jensen <[email-masked]>
Date: Mon, 05 Feb 2001 14:14:47 -0700

> dma_cache_invalidate() (or some functions to that effect) at certain places.

Yes, the tulip driver calls pci_unmap_single() on the receive
buffer, but for mips (in asm-mips/pci.h) this call does
nothing.  And this is what is so confusing.  Only if the
receive buffer was forced to be in KSEG1 would this make
sense.

> What is the CPU?  It seems logical to suspect about the dma cache routines.

Yes, I have scrubbed over my patch to the cache routines
many times, especially since on the IDT 334 the cache-way
selection for indexed cache ops is weird--they left the
way-bit up at bit 12 as if it were an 8KB cache, when
in reality it is only a 2KB cache.

Quinn

---

From: Kenneth C Barr <[email-masked]>
Date: Mon, 5 Feb 2001 16:13:23 -0500

Thank you very much for your help so far!  Here's an update and another
question:

Updates:
1.  Had to attach a console (we have XZ graphics not XL)
2.  An ecoff kernel image downloaded fine.  The ELF ones download halfway
and fail silently (even with the console attached)

Question: As the vmlinux-indy-2.2.1-990226.ecoff kernel begins the boot
sequence, it hangs at "unable to open initial console."  I had unpacked
the filesystem on linux as the FAQ suggests, and my device files look
right.  So, on the NFS server, I symlinked /dev/console to /dev/ttyS0.
Now the message is replaced with a cryptic: "S<<<<<"

Perhaps that kernel doesn't have console support?  Is there any other
explanation for the "S<<<<<" failure where I used to see "...initial
console?"

I'd like to try a different kernel but I can't find an elf2ecoff binary.
Does anyone have this converted or a known-good ecoff kernel?

Ken

---

From: Alan Cox <[email-masked]>
Date: Mon, 5 Feb 2001 21:39:51 +0000 (GMT)

> wrong value the driver should work.

If you set it wrongly you can get network failures/stalls. The normal cases
are not a problem however. You need to copybreak small frames to avoid filling
the socket queue with empty spaces. Some platforms set it to 1518 to force
a copy thereby aligning the IP header on a 4 byte boundary, which most
PCI bus master drivers wont do if you are not copying

---

From: Geert Uytterhoeven <[email-masked]>
Date: Tue, 6 Feb 2001 07:32:53 +0100 (MET)

> Now the message is replaced with a cryptic: "S<<<<<"

However, /dev/console must _not_ be a symlink, but a character special device
with major 5 minor 1 (cfr. Documentation/devices.txt).

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven ------------- Sony Software Development Center Europe (SDCE)
[email-masked] ------------------- Sint-Stevens-Woluwestraat 55
Voice +32-2-7248626 Fax +32-2-7262686 ---------------- B-1130 Brussels, Belgium

---

From: Klaus Naumann <[email-masked]>
Date: Tue, 6 Feb 2001 19:07:47 +0100 (CET)

> with major 5 minor 1 (cfr. Documentation/devices.txt).

Unless you're using Linux/MIPS :)
The console implementation is horrible broken, so
this means that under most circumstances the 5,1 device doesn't
work. So the only option is to make /dev/console a symlink to /dev/ttyS0.

		Cya, Klaus

-- 
Full Name   : Klaus Naumann     | (http://www.mgnet.de/) (Germany)
Nickname    : Spock             | Org.: Mad Guys Network
Phone / FAX : ++49/177/7862964  | E-Mail: ([email-masked])
PGP Key     : www.mgnet.de/keys/key_spock.txt

---

From: Florian Lohoff <[email-masked]>
Date: Tue, 6 Feb 2001 21:24:07 +0100

> work. So the only option is to make /dev/console a symlink to /dev/ttyS0.

This has nothing to do with "Linux/Mips" and is definitly not true
on all subarchs. It might be true on SGI machines but i dont even
there its necessary.

Flo
-- 
Florian Lohoff                  [email-masked]             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?