Subject: A question about architecture and byte order with RPMs

From: Alex deVries <[email-masked]>
Date: Thu, 4 Dec 1997 00:31:05 -0500 (EST)

Below is a message I sent off to the RPM development mailing list. The
folks at RedHat said it was reasonable, but I just wanted to be sure that
I got it right. Many of you understand MIPS architectures better than I,
and we don't want to be making a mistake.

Is the creation of a mipsel type reasonable?

- Alex

---------- Forwarded message ----------
Date: Tue, 2 Dec 1997 11:34:54 -0500 (EST)
From: Alex deVries <[email-masked]>
To: RPM-List <[email-masked]>
Subject: A question about architecture and byte order.

I *think* there might be an issue with MIPS architecture RPMs, but I want
to make sure I get things right.

There are two branches of machines that have MIPS processors.  The first
is little endian, and it contains things like Acer Pica and Mips Magnum.
The second is big endian, and has things like my SGI Indy[1]. I'm unclear
if there are some architectures that will run both.

Now, the issue is that there aren't distinct architecture definitions for
mips (big endian) and mipsel (little endian). They aren't binary
compatible, so it does seem to me that there should be an entry like:

arch_canon:	mipsel:	mipsel	11

in rpmrc. 

Am I wrong on this?

- Alex

[1] *almost* running srpms of RedHat 5.

      Alex deVries          Rent this space for a $5 donation 
  System Administrator      to EngSoc per day.
   The EngSoc Project       Send spam to [email-masked].

---

From: [email-masked] (Ariel Faigon)
Date: Thu, 4 Dec 1997 00:10:34 -0800 (PST)

You are correct except the "mips" (big-endian) on SGI situation
it is even more complex :-)

Come gcc 2.8 and SGI/Linux could theoretically support
N32 (the newer MIPS 32-bit ABI) and N64 (64 bits)
Just like the IRIX native cc compiler.

Unlike the Alpha which has only one programming model
	sizeof(long) = sizeof(char *) = 8

--- cut and save ---
SGI has 2 programming models (source interpretation):

	1) 64 bit (aka N64 or 64)
	   compiled using "cc -64"

		sizeof(long) = sizeof(char *) = 8
		sizeof(int) = 4

	2) Old 32-bits (aka o32) compiled using "cc -32"
		And
	   New 32-bits (aka n32) compiled using "cc -n32"

		sizeof(long long) = 8
		sizeof(int) = sizeof(long) = sizeof(char *) = 4

Going into binaries: o32 and n32 are not the same, the code
generated by n32 uses more efficient calling conventions
and more registers and cannot run on old versions of the OS.
New hardware and versions of the OS can run all of them except
if your 'uname' says 'IRIX' and not 'IRIX64' it can compile
(but not run) N64 binaries.

The advantages of this is that the new models run faster
since they are utilizing the new hardware better, plus
that you don't need to make all the apps 64-bits when
only 1% of them actually need this (a big saving in code
and data size).

The disadvantage is complexity and confusion...
Once SGI/Linux becomes stable, if gcc 2.8 is out and stable
we could try and migrate to supporting n32 (the dynamic linker
should support it, and a new set of dynamic libraraies should
be built for this)

So on SGI RPM should probably be named:
	mips-<abi>.rpm

where <abi> is one of:
	o32
	n32
	64

Currently SGI/Linux is purely o32 since that is the only ABI
gcc 2.7.x knows about.  The downside is that it is less efficient
than what it could have been (not utilizing the R4xxx and up
processors really well).

--- cut and save ---

:
:
:Below is a message I sent off to the RPM development mailing list. The
:folks at RedHat said it was reasonable, but I just wanted to be sure that
:I got it right. Many of you understand MIPS architectures better than I,
:and we don't want to be making a mistake.
:
:Is the creation of a mipsel type reasonable?
:
:- Alex
:
:---------- Forwarded message ----------
:Date: Tue, 2 Dec 1997 11:34:54 -0500 (EST)
:From: Alex deVries <[email-masked]>
:To: RPM-List <[email-masked]>
:Subject: A question about architecture and byte order.
:
:
:I *think* there might be an issue with MIPS architecture RPMs, but I want
:to make sure I get things right.
:
:There are two branches of machines that have MIPS processors.  The first
:is little endian, and it contains things like Acer Pica and Mips Magnum.
:The second is big endian, and has things like my SGI Indy[1]. I'm unclear
:if there are some architectures that will run both.
:
:Now, the issue is that there aren't distinct architecture definitions for
:mips (big endian) and mipsel (little endian). They aren't binary
:compatible, so it does seem to me that there should be an entry like:
:
:arch_canon:	mipsel:	mipsel	11
:
:in rpmrc. 
:
:Am I wrong on this?
:
:- Alex
:
:[1] *almost* running srpms of RedHat 5.
:
:
:      Alex deVries          Rent this space for a $5 donation 
:  System Administrator      to EngSoc per day.
:   The EngSoc Project       Send spam to [email-masked].
:
:
:
:
:

-- 
Peace, Ariel

---

From: [email-masked]
Date: Thu, 4 Dec 1997 11:43:48 +0100

> Is the creation of a mipsel type reasonable?

Definately, only the fact that there are more important things to do did
so far keep me from doing so.

> if there are some architectures that will run both.

While almost all MIPS CPUs can run in both byteorder some systems like
the Acer Pica or DECstations don't support this CPU feature.  There is
still another CPU feature that allows to run the other flavor of software
in usermode but supporting it not a Sunday afternoon hack.  It requires
going through _all_ the kernel code and possibly fixing the byteorder
handling.

> arch_canon:	mipsel:	mipsel	11

Unless the rpm gurus think there is a better way to do things - yes, it
looks right to me.  The other thing that needs to be done is to teach
rpm how to recognice the various system flavours.  Currently rpm relies
on uname() returning "mips" and therefore thinks MIPS is always MIPS.

Have to take a look at the rpm sources - is there an official way to
teach rpm about incompatible variants of the same CPU architecture?

Modifying uname() to return "mipsel" etc. is a bad choice.  For most
software it is more important to know the CPU architecture than certain
specialised details like the byteorder.  So returning "mipsel" would
break a hell lot of software.

  Ralf

---

From: "William J. Earl" <[email-masked]>
Date: Thu, 4 Dec 1997 08:42:11 -0800

Ariel Faigon writes:
...
 > than what it could have been (not utilizing the R4xxx and up
...

      There is a further complication.  MIPS R3000-based systems will
only run o32 in the MIPS I subset.  MIPS R6000-based systems will
run o32 in MIPS I or II.  On R4000 and newer systems (including
R4400, R4600, R4700, R5000, R8000, and R10000), o32 can be any of MIPS I,
II, or III, and can be MIPS IV on R5000, R8000, or R10000, although
MIPS III and IV are not officially supported by SGI systems for o32.
n32 and n64 can be either MIPS III or MIPS IV, with MIPS IV supported
only on R5000, R8000, and R10000.  n32 is not really better than o32
for integer programs; it is somewhat better for floating point programs
which need a lot of registers, since it enables the use of all 32 64-bit
floating point registers, instead of the MIPS I-compatible 16 registers.
n64 is the same as n32, except for having 64-bit addresses and long
integers.  Note that all models support "long long" 64-bit integers,
on all versions of the architecture, although they are emulated using
32-bit arithmetic when compiling for MIPS I and II.

     I would recommend that "mips.rpm" be the default o32 built for
MIPS I, perhaps with selected DSOs built both MIPS I and MIPS III or
IV (multiple versions), with the appropriate version linked to the
standard name at installation time, for those DSOs where the
architecture changes make a difference.  That is, one could have
libgl.so being a symbolic link to one of libgl-mips1.so,
libgl-mips3.so, or libgl-mips4.so, with the default (and initial
setting to guarantee booting on all systems) being to link to
libgl-mips1.so.  One might also add extra runtime libraries
(in an optional package) for n32, but the gains are very small
(and the code size larger) for most normal libraries and applications.

     n32 really makes a difference for SPECfp (FORTRAN) programs.  It
rarely makes much difference for integer programs.  On SGI systems,
cc -n32 now often produces better code than cc -o32, but that is because
they are really different compilers, and cc -o32 has had almost no work
done on it for about 5 years.  Both typically produce better code than
gcc, at least at high optimization levels, as measured by SPECint
performance.

     MIPS III does not help o32 programs for the most part, unless they
do a lot of 64-bit (long long) integer arithmetic.  MIPS IV adds some
useful floating point instructions and integer conditional moves.  The
floating point instructions include madd and the like, and those make
a large difference for graphics code and for DSP-like algorithms.
Neither make a major difference to the operating system or libc,
so most o32 libraries might as well be MIPS I, except perhaps for
a few routines such as bcopy, which can be much faster in MIPS III.

---

From: Alex deVries <[email-masked]>
Date: Thu, 4 Dec 1997 11:45:41 -0500 (EST)

> (but not run) N64 binaries.

The architecture of the binary RPM concerns itself with the CPU only.
>From what I understand in your explanation, supporting o32, n32 and 64
executables is a function of the OS and the libraries.  RPM binaries take
care of this already with their OS name field and their libraries.

So, I don't think we'd need to concern ourself with the o32, n32 and 64
options within the architecture field.

Does this make sense? It's important that we don't screw ourselves over.

- Alex

---

From: Alex deVries <[email-masked]>
Date: Thu, 4 Dec 1997 12:04:36 -0500 (EST)

> the Acer Pica or DECstations don't support this CPU feature.  There is

Right.  So they'd be of arch 'mips'.

> in usermode but supporting it not a Sunday afternoon hack.  

Hm.  RPM makes no diffrentiation between user and non-usermode apps.  So,
I'd say we'd mark this as 'mipsbi'.  The only restriction is with kernel
RPMs, and there aren't a lot of those.

> It requires

Nobody wants that.

> on uname() returning "mips" and therefore thinks MIPS is always MIPS.

That shouldn't be too difficult to build into .configure.  For everything
matching mips:*, configure should do a byte order execution check. If it
can do mipsel, it's little endian.  If it can do bigendian, it's mips.  If
it can do both, it's mipsbi.  Is there an easy way to do a test like
this, Ralf?

> teach rpm about incompatible variants of the same CPU architecture?

Yes, sort of.

There should be two binary formats defined in /usr/lib/rpmrc
arch_canon: mips: mips 4 (already defined)
arch_canon: mipsel: mipsel 11 

Then, 
arch_compat: mips: noarch 
arch_compat: mipsel: noarch 
arch_compat: mipsbi: mips mipsel

> Modifying uname() to return "mipsel" etc. is a bad choice.  For most

I agree.  I think we should just patch the configure script to do a
translation to mips, mipsel or mipsbi, as I described above.

- Alex

---

From: "Greg Chesson" <[email-masked]>
Date: Thu, 4 Dec 1997 09:17:39 -0800

On Dec 4, 12:04pm, Alex deVries wrote:
> Subject: Re: A question about architecture and byte order with RPMs

> > going through _all_ the kernel code and possibly fixing the byteorder

I've been asking the compiler people for a storage class modifier that could
be added to type declarations.  The modifier would indicate whether the
data object is stored in big-endian or little-endian format.
The compiler would generate byte swizzles or not, depending on whether
the native execution mode agrees with the indicated storage class.

Perhaps the GCC world, being somewhat more enlightened, could do something
in this area (or perhaps already is thinking about it?).

g

---

From: "Ron G. Minnich" <[email-masked]>
Date: Thu, 4 Dec 1997 13:40:15 -0500 (EST)

> the native execution mode agrees with the indicated storage class.

or you could always look at the gokhale/minnich paper from 1993 (SEDMS) in
which we built a compiler that did just that, conforming to the IEEE
1596.5 standard (which I helped get going along with David James). It's 
not that hard.

We did a bit more than that, maybe too much more, but still ...

I still think it's a good idea :-)

ron

---

From: [email-masked] (Alan Cox)
Date: Thu, 4 Dec 1997 18:56:05 +0000 (GMT)

> in this area (or perhaps already is thinking about it?).

Yeuch ;). Can't you get your guys to put that in the MMU so you can have
a little endian or big endian page ? I can see how you'd make lcc generate
such output but not offhand gcc

---

From: "Greg Chesson" <[email-masked]>
Date: Thu, 4 Dec 1997 11:42:21 -0800

Future MMUs are in the future.  I'm more interested in solving the problem
in a device-independent way.  Also, doing it on a page basis is a bit
restrictive and limits your options.

g

---

From: [email-masked]
Date: Fri, 5 Dec 1997 00:39:43 +0100

> restrictive and limits your options.

Basically I think that the reverse endian bit as it is implemented in
the kernel isn't too useful.  As things are in the EntryLo register
pair on R4000 and better no bit to do things on a per page base as
Alan suggests is left.  Well, at lest not in the lower six bits that
currently contain the control information.  If the reverse endian
bit would also reverse accesses to userspace when running in kernel
mode, it'd be much better for performance as most of the byteswapping
could be left out.  Ok, this in the hope some silicon guys are reading
on this list. (Are there any?)

As far as your original suggestion goes - yes, there is a way to declare
variable attributes in GCC.  It works like:

  unsigned int zumbitsu __attribute__((unaligned));

This for example tells GCC that the variable zumbitsu may not be
allocated correctly aligned and it should generate code that takes
care of that.  Goodbye, address error :-)  This stuff is documented
in the GCC info pages, search for ``variable attributes'' and
``function attributes''.

Extending that mechanism to do byteorder swapping shouldn't be too
hard.  I however don't believe this to be a too useful mechanism
for most applications because a compiler doesn't know if used in
a particular context struct foo has to be swapped or not.

Did you propose this mechanism for Risc/OS 6?

  Ralf

---

From: "Ron G. Minnich" <[email-masked]>
Date: Thu, 4 Dec 1997 21:39:14 -0500 (EST)

> such output but not offhand gcc

If it's in the compiler:
1) htonx and ntohx are history
2) xdr is history

not a bad deal in my book. Put it in the MMU? 2^yeuch. :-)
ron

---

From: "David S. Miller" <[email-masked]>
Date: Thu, 4 Dec 1997 18:59:51 -0800

Date: Thu, 4 Dec 1997 21:39:14 -0500 (EST)
   From: "Ron G. Minnich" <[email-masked]>

   If it's in the compiler:

Some architectures (I believe both PPC and UltraSPARC) provide three
mechanisms to specify endiannes for load and store operations.

1) In processor status register, a "everything foo-endian" bit
   exists, there is also a "trap foo-endian" bit which states
   which endianness exception handlers run with.

2) Load and Store instructions can specify "endianness attributes"
   so for example on Sparc-V9 you can say

   lda	  [%addr_reg] ASI_PL, %dest_reg

   ASI means "Address Space Identifier" and PL means "Primary address
   space Little-endian".

3) Per-page MMU endianness attribute bits.  Those who casually
   disassemble the solaris kernel now and again will notice that the
   SLAB allocator in UltraSPARC kernels provides two sets of
   interfaces for allocation, one gives you little endian chunks the
   other gives you big-endian chunks.  They implement this by frobbing
   the PTE bits in the kernel mappings for these SLABs.

In the age of PCI, these three mechanism are super handy for
Big-Endian systems.  You get the best of both worlds, of interest
namely is:

1) No byte swapping for IP networking header processing.

2) Direct access to legacy PCI cards, in the correct PCI
   (ie. little endian) byte order, at zero cost.

3) Support for arbitrary endianness filesystem layouts, in
   core, again at zero cost.

Later,
David S. Miller
[email-masked]