Subject: load_unaligned() and "uld" instruction

From: Jun Sun <[email-masked]>
Date: Mon, 25 Sep 2000 11:48:28 -0700

The USB sub-system uses "unaligned.h" file to access unaligned data. 
All the unaligned data access functions depend on "uld" and "usw"
instructions, which are not available on many CPUs.

I wonder if there is a version of unaligned access functions which do
not depend on those instructions.  If not, I can probably write one.

Any suggestions?

Jun

---

From: Dominic Sweetman <[email-masked]>
Date: Mon, 25 Sep 2000 22:16:33 +0100 (GMT/BST)

> instructions, which are not available on many CPUs.

You won't find the instruction 'uld' in *any* MIPS CPU.

uld is an assembler macro-instruction translating into a 

  ldl
  ldr

pair (the instructions are called load-double-left and
load-double-right).  The exact translation depends on whether you're
running big-endian or little-endian... but the 32-bit version on a
big-endian CPU is that 

  ulw $1, <address>

is assembled as

  lwl $1, <address>
  lwr $1, <address+3>

The way that the load-left and load-right work together is kind of
tricky to get your head round.  

So far as I know, all 64-bit MIPS CPUs implement ldl/ldr and the store
equivalents.  MIPS patented these instructions, so clones like Lexra's
don't implement the 32-bit versions (lwl, lwr etc).

-- 
Dominic Sweetman
Algorithmics Ltd
The Fruit Farm, Ely Road, Chittering, CAMBS CB5 9PH, ENGLAND
phone: +44 1223 706200 / fax: +44 1223 706250 / http://www.algor.co.uk

---

From: Jun Sun <[email-masked]>
Date: Mon, 25 Sep 2000 14:36:39 -0700

> don't implement the 32-bit versions (lwl, lwr etc).

Dominic,

Thanks for the clarification.

I looked at my problem again, and it turns out that it was caused by
"-mips2" compiler option.  If I use "-mips3", the complain goes away,
which seems to make sense - assuming "uld" and "usw" are introduced in
mips III.

This actually brings another question (which I thought I have posted
before).  Take a look of arch/mips/Makefile, you will find most CPUS
uses -mips2 compiler option.  While -mips2 is safe, it cannot take
advantages of "uld" etc.  Is there any reason that we don't want to use
-mips3, at least for some of the later CPUs?

If we have to use "-mips2" option, is there a clean way which allows us
to "uld/usw" instructions (instead of manually twicking the compilation
for each file that uses them)?

Another question is that in the same file most CPUs will take another
compiler option such as "-mcpu=r8000", in which case the cpu model
usually does NOT correspond to the actual CPU.  Why is that?

Thanks.

Jun

---

From: Ralf Baechle <[email-masked]>
Date: Tue, 26 Sep 2000 01:29:22 +0200

> -mips3, at least for some of the later CPUs?

You cannot use any kind of 64-bit operation for the 32-bit kernel except
for the $zero register.  This is because all exceptions as far as they
store / restore the integer registers at all will only deal with the lower
32-bit of the registers.  In other word any interrupt will corrupt the
upper 32-bit bit of gp registers.

Back in history I tried to enable the use of the full 64-bit register in
the kernel - it ended up ugly as hell, especially because we still want
to be able to share most of the code with the R3000.

> usually does NOT correspond to the actual CPU.  Why is that?

-mcpu=<somecpu> chooses what CPU gcc will schedule instructions for.  No
matter what value you choose for <somecpu> the code will run on all CPUs.
-mips<n> chooses which ISA level gcc will generate code for; that code
won't run on CPUs with a ISA level less than <n>.

  Ralf

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Tue, 26 Sep 2000 08:22:36 +0200

> Thanks for the clarification.

I'll second that - he beat me to it!

> which seems to make sense - assuming "uld" and "usw" are 
> introduced in mips III.

The "load word left/right" and "store word left/right" instructions are 
part of the original MIPS I ISA.  On the other hand, "uld" represents
a load of an unalgined quad or "doubleword" of 64-bits, and uses
64-bit load double right/left instructions that are part of the 64-bit
MIPS III ISA.  

> for each file that uses them)?

This is a general problem that I've had to fight with the 
"main line" MIPS/Linux distribution.  Most of the work
being done is being done on SGI platforms, and all
SGI systems since the Crimson have had 64-bit CPUs.
Older DECStations use R3000s, and more importantly,
many of the new embedded MIPS designs use "MIPS32"
processors that have R4000-like system coprocessors,
but only 32-bit data paths.  I had to do a fairly complete
redesign of the 2.2 semaphore support code, for example,
in order to get it to rely only on the 32-bit forms of load
locked and store conditional.  It's clear that I'll have to do
something similar with the unaligned accesses in the USB 
support code before it will run on the MIPS 4Kc and 
similar CPUs.

> usually does NOT correspond to the actual CPU.  Why is that?

The -mcpu tells the compiler and assembler for what kind
of pipeline it should optimise, which is independent of the
ISA level.  "-mcpu=r8000", for example, tells the tools that
the CPU is superscalar. Thus one sees that option selected 
for the R5000 platforms, even though the R5000 and R8000
pipelines are otherwise very dissimilar.

            Regards,

            Kevin K.

---

From: Dominic Sweetman <[email-masked]>
Date: Tue, 26 Sep 2000 10:08:15 +0100 (GMT/BST)

> pipelines are otherwise very dissimilar.

Hmm.  I wish it was that simple.  But some MIPS CPUs have 
instruction set additions which are not related to the mips1, mips2,
etc.  For example, a whole collection of parts with a vaguely
"embedded" orientation has integer multiply/accumulate instructions.

Algorithmics' version of GCC (and, I'm sure, others) picks up on the
-mcpu=xxx flag to do that.  In fact, I don't think there's any other
way to allow the compiler to warn you of some bizarre omissions from
one or two rogue CPUs.

But until compiler support for MIPS Linux is more systematic, you'd be
better being conservative.  And you don't want to unnecessarily
multiply kernel versions - so in general, don't say "-mcpu=" anything
for kernel builds.

The Linux convention is "-mips2"; which is quite odd, because the
MIPS-II ISA was incarnate in just one CPU (the R6000).  A few units
were made around 1990 and even fewer worked; the project was overtaken
by the (-mips3, 64-bit) R4000.

Subsequently, and confusingly, "-mips2" has been re-used to mean
"-mips3 but don't assume 64-bit registers".  Except for floating
point.  Maybe.  (it's sometimes not a good idea to re-use a term).

Ralf wrote:

> You cannot use any kind of 64-bit operation for the 32-bit kernel...

Outside SGI circles, I believe, "32-bit kernels" are all that are
likely to work...

> ... except for the $zero register.  This is because all exceptions
> as far as they store / restore the integer registers at all will
> only deal with the lower 32-bit of the registers.  In other word any
> interrupt will corrupt the upper 32-bit bit of gp registers.

Even calling a subroutine compiled 32-bit may corrupt one of the
registers which are supposed to be preserved.

As Kevin indicates, it would probably be worth some effort to converge
on a kernel which would:

1. build for either 32-bit ("MIPS32" and near-miss) and 64-bit
  (MIPS3, MIPS4 and MIPS64) CPUs.

2. Allow 64-bit operations on 64-bit CPUs, without insisting that
   C data types grow.  Need to save the whole of registers and compile
   "long long" and "double" data types...

This is possible, but needs some thought.  AFAIK, the GCC currently
used for Linux changes the whole calling convention when -mips3 is
selected, which makes (2) pretty difficult.

---

From: Jun Sun <[email-masked]>
Date: Tue, 26 Sep 2000 11:04:12 -0700

> > for each file that uses them)?

Ralf, before the perfect solution is found, the following patch makes
the gcc complain go away.  It just use ".set mips3" pragma.
 
> It's clear that I'll have to do

I am pretty close to get USB running with the v2.4-test5.  The unaligned
access is the minor problem.  The bigger problem I am fighting with now
is bus_to_virt()/virt_to_bus() and USB interrupt.

Jun

=====================================

--- linux/include/asm-mips/unaligned.h.orig     Mon Sep 25 14:02:52 2000
+++ linux/include/asm-mips/unaligned.h  Tue Sep 26 10:53:31 2000
@@ -19,7 +19,7 @@
 {
        unsigned long long __res;
 
-       __asm__("uld\t%0,(%1)"
+       __asm__(".set\tmips3\n\tuld\t%0,(%1)"
                :"=&r" (__res)
                :"r" (__addr));
 
@@ -33,7 +33,7 @@
 {
        unsigned long __res;
 
-       __asm__("ulw\t%0,(%1)"
+       __asm__(".set\tmips3\n\tulw\t%0,(%1)"
                :"=&r" (__res)
                :"r" (__addr));
 
@@ -47,7 +47,7 @@
 {
        unsigned long __res;
 
-       __asm__("ulh\t%0,(%1)"
+       __asm__(".set\tmips3\n\tulh\t%0,(%1)"
                :"=&r" (__res)
                :"r" (__addr));
 
@@ -60,7 +60,7 @@
 extern __inline__ void stq_u(unsigned long __val, unsigned long long *
__addr)
 {
        __asm__ __volatile__(
-               "usd\t%0,(%1)"
+               ".set\tmips3\n\tusd\t%0,(%1)"
                : /* No results */
                :"r" (__val),
                 "r" (__addr));
@@ -72,7 +72,7 @@
 extern __inline__ void stl_u(unsigned long __val, unsigned int *
__addr)
 {
        __asm__ __volatile__(
-               "usw\t%0,(%1)"
+               ".set\tmips3\n\tusw\t%0,(%1)"
                : /* No results */
                :"r" (__val),
                 "r" (__addr));
@@ -84,7 +84,7 @@
 extern __inline__ void stw_u(unsigned long __val, unsigned short *
__addr)
 {
        __asm__ __volatile__(
-               "ush\t%0,(%1)"
+               ".set\tmips3\n\tush\t%0,(%1)"
                : /* No results */
                :"r" (__val),
                 "r" (__addr));

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Wed, 27 Sep 2000 12:06:31 +0200 (MET DST)

> +++ linux/include/asm-mips/unaligned.h  Tue Sep 26 10:53:31 2000
[etc.]

 Please don't.  Gcc already has means to generate proper unaligned
accesses.  See include/asm-alpha/unaligned.h for how to achieve them in a
portable way (i.e. using packed structs) without the problematic inline
asm.

 And please use ".set mips0" (or ".set push" and ".set pop",
appropriately) after using any ".set mips*" directive (or any other ".set"
directive to that matter) not to adversly affect any other code.  Improper
coding of such constructs bites R3K people badly.

 Better yet, configure your compiler appropriately and avoid switching ISA
levels in the code if at all possible.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 29 Sep 2000 19:22:54 +0200

> way to allow the compiler to warn you of some bizarre omissions from

Ouch.  The gcc documentation says this:

`-mcpu=CPU TYPE'
     Assume the defaults for the machine type CPU TYPE when scheduling
     instructions.  The choices for CPU TYPE are `r2000', `r3000',
     `r4000', `r4400', `r4600', and `r6000'.  While picking a specific
     CPU TYPE will schedule things appropriately for that particular
     chip, the compiler will not generate any code that does not meet
     level 1 of the MIPS ISA (instruction set architecture) without the
     `-mips2' or `-mips3' switches being used.

So in other words I wouldn't expect anything like mmad to be used unless
-mmad is also being choosen.  -mcpu not influencing the set of instructions
being used to build a program is a general gcc convention, not only for
MIPS.  So if the Algorithmics compiler does things different I'd consider
it to be off the track.

> point.  Maybe.  (it's sometimes not a good idea to re-use a term).

In the kernel we actually don't care very much about floating point.

> Outside SGI circles, I believe, "32-bit kernels" are all that are

Currently.  Some embedded people are actually asking for more than the
512mb memory supported by the 32-bit kernel.  So expect the 64-bit
kernel to become the predominant race in the not to distant future.
Also expect embedded SMP kernels in the not to far future.

No, I don't feel at all like adding highmem support to the 32-bit kernel.

> registers which are supposed to be preserved.

Sure, but that's kind of expected and obvious when following the
instruction sequence as it gets executed while the corruption by an
exception was pretty unobvious when I first ran into it ...

>    "long long" and "double" data types...

I was thinking about moving all the 64-bit CPUs over to the mips64 kernel
and leave the `mips' kernel to the true 32-bit stuff.  If you go and
download a 2.0.14 tarball you'll see that I already once tried to support
full 64-bit operation but only 32-bit address space altogether with
real 32-bit CPUs in the `mips' architecture.  The result was fairly ugly,
so having learned form that I would prefer to keep 32-bit and 64-bit
stuff separate.

Most users will currently still not want to use a 64-bit address space
for apps.  That's ok, we can add support for 2-level page tables to
`mips64'.  That's already been done for example for x86 and looks
fairly sane and maintainable.

> selected, which makes (2) pretty difficult.

The calling conventions used by -mips3 are slight confusing, if not even
dangerous.  Older gccs use a non-standard calling convention which essentially
is a blind extension of the 32-bit ABI to 64-bit.  Newer gccs support
the N32 and 64 ABIs.  Unfortunately currently gcc does not support building
a single compiler that supports all three 32, N32 and 64 ABIs.

  Ralf