Subject: Multiple processor support?

From: "Matthew Dharm" <[email-masked]>
Date: Fri, 23 Mar 2001 14:53:59 -0800

Does the MIPS port of linux support multiple-processor architectures?

Matt Dharm

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

---

From: Justin Carlson <[email-masked]>
Date: Fri, 23 Mar 2001 14:58:20 -0800

> Does the MIPS port of linux support multiple-processor architectures?

MIPS or MIPS64?

-Justin

---

From: "Matthew Dharm" <[email-masked]>
Date: Fri, 23 Mar 2001 16:01:26 -0800

Well, I'd like to know about both, frankly.  Tho I'm more interested
in whichever is designed to run on RM7000 series processors.

Matt

> -----Original Message-----
> From: Justin Carlson [mailto:[email-masked]]
> Sent: Friday, March 23, 2001 2:58 PM
> To: Matthew Dharm
> Cc: [email-masked]
> Subject: Re: Multiple processor support?
>
>

---

From: Justin Carlson <[email-masked]>
Date: Fri, 23 Mar 2001 16:08:13 -0800

> in whichever is designed to run on RM7000 series processors.

To the best of my knowledge, the mips64 tree only works in SMP on the ip-27
which is r10K based.  There would be a bit of work to get an RM7K  based
multiprocessor system to run. A fair amount of the "generic" code in
that tree is also pretty ip-27 specific, and so would need to be cleaned up.

I'm working on mips32 SMP support at the moment; there are no existing ports of
this tree to an SMP platform.  The mips64 stuff is certainly much, much more 
mature.  I don't know of any reasons not to use the mips64 side for an RM7K.

-Justin

---

From: Joe George <[email-masked]>
Date: Fri, 23 Mar 2001 17:13:48 -0700

Unfortunately the RM7000 does not have hardware cache coherency
support.

Joe George

Matthew Dharm wrote:

>>> Does the MIPS port of linux support multiple-processor
>> 
>> architectures?
>> 
>> MIPS or MIPS64?
>> 
>> -Justin
>> 
>>

---

From: Keith M Wesolowski <[email-masked]>
Date: Fri, 23 Mar 2001 16:29:05 -0800

> that tree is also pretty ip-27 specific, and so would need to be cleaned up.

In the particular case of SMP, this is probably still true.  However,
as of last weekend this tree compiles properly and boots part way on
Indy; I have more to do but the separation of the obviously
ip27-dependent stuff is done; you should at least be able to compile
other machines in mips64...if there were any.

-- 
Keith M Wesolowski <[email-masked]> http://foobazco.org/~wesolows
------(( Project Foobazco Coordinator and Network Administrator ))------
"I should have crushed his marketing-addled skull with a fucking bat."

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Sat, 24 Mar 2001 01:40:47 +0100

> > in whichever is designed to run on RM7000 series processors.

None are "designed" as such for the RM7000.  I don't have a
full RM7000 manual in the shop, but the short-form advance
manual that I do have, while it goes into a fair amount of
detail about the cache operations and external interface
protocols, makes no mention of any support for hardware
coherence of the sort provided by the R10000/R12000
(and the R4000/R4400 for that matter).  If there is no support
for cached/coherent memory attributes and cache interventions
from the system side, an SMP kernel design for an MP SGI box
might not be useful for an MP RM7K configuration.  It is possible,
but tricky, and at times unavoidably inefficient to build a
software-coherent SMP system.  I have not heard of anyone
doing so with MIPS/Linux.

> mature.  I don't know of any reasons not to use the mips64 side for an
RM7K.

Well, one reason might be memory footprint...

            Regards,

            Kevin K.

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 24 Mar 2001 02:23:07 +0100

> Well, one reason might be memory footprint...

As of now the memory footprint of the kernel is large but userspace which
is all 32-bit software has unchanged footprint.  I've got plans for 2.5
to reduce the memory footprint of the kernel by introducing a 2-level
pagetable.

  Ralf

---

From: Jun Sun <[email-masked]>
Date: Mon, 26 Mar 2001 11:20:35 -0800

"Kevin D. Kissell" wrote:
> 
> (Software cache coherency) It is possible,

How would it be possible?  Any reference to the previous implementations?

I imagine you would need at least some kind of atomic operation (like ll/sc)
working reliably (which itself may require cache coherency).  Also, any such
scheme should not require massive change in the programming.

I am very curious....

Jun

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Thu, 29 Mar 2001 11:16:41 +0200

> How would it be possible?  Any reference to the previous implementations?

Lots of work on software coherent schemes was done in the
mid-late 1980s.  Check out the ASPLOS, and ISCA proceedings
from the period for references.  In essence, such schemes involve
the identification of critical regions at risk, the use of barriers around
such regions, and an explicit cache flush/purge protocol.  You can think
of the more common MP "TLB shootdown" protocols as being a variant
of a software cache coherence scheme.

> working reliably (which itself may require cache coherency).

MIPS ll/sc, as defined and implemented, does require hardware
coherency support for correct multiprocessor operation.  But one
can, in principle, construct a software-coherent SMP system even
in the absence of such a primitive - many of the implementations
of software coherent SMPs used software coherence precisely
because they were based on simple switch/crossbar interconnects
where snooping was not possible.

> Also, any such

Whether progams need to change depends on the coherency
and consistency models assumed by the program.  Certainly
a naive multithreaded program that assumes an SGI-like model
could not be dropped onto a software-coherent MP system without
recompilation with specialized compilers at a minimum, and
more likely not without recoding.  On the other hand, if one's objective
is to run multiple, independent programs on different CPUs in
an SMP system, it should only be the OS that should need to
change to deal with the coherence issues for shared user pages
and shared kernel data structures, and to ensure that any
multithreaded application that is not explicitly set up to handle
software cache coherency has its threads bound to the same
CPU and caches (defeats some of the point of having a
multithreaded program, I know, but...).

            Regards,

            Kevin K.