Subject: Cross compiling RPMs

From: Mike McDonald <[email-masked]>
Date: Fri, 26 Jan 2001 10:15:21 -0800

Can anyone point me to some references of techniques for cross
compiling RPMs? I want to build some packages for my little endian
MIPS but I haven't found any info on cross compiling RPMs in the RPM
docs nor "Maximum RPM". Any pointers would be appreciated. (I'm
particularly interested in how to specify the tool chain.)

  Thanks,

  Mike McDonald
  [email-masked]

---

From: Pete Popov <[email-masked]>
Date: Fri, 26 Jan 2001 10:37:03 -0800

> particularly interested in how to specify the tool chain.)

To start with, you'll need a cross tool chain setup properly with the
headers and libraries.  One option is
ftp.mvista.com:/pub/Area51/mips_fp_le. You can grab everything (the
entire root fs) or just the tools: binutils, gcc, kernel headers,
glibc.  Others might have similar toolchains they can point you at. 
Another option is native builds, which I personally don't like.

Pete

---

From: Mike McDonald <[email-masked]>
Date: Fri, 26 Jan 2001 11:27:48 -0800

>Date: Fri, 26 Jan 2001 10:37:03 -0800
>From: Pete Popov <[email-masked]>
>To: Mike McDonald <[email-masked]>
>Subject: Re: Cross compiling RPMs

>Another option is native builds, which I personally don't like.

  I have a working tool chain that I use to cross compile a kernel
with sources from. How do I convince rpm to use that chain?

  Mike McDonald
  [email-masked]

---

From: Pete Popov <[email-masked]>
Date: Fri, 26 Jan 2001 11:39:17 -0800

> with sources from. How do I convince rpm to use that chain?

Is that tool chain setup to compile userland apps? Can you cross compile
this:

hello.c:

int main()
{
    printf("hello world\n");
}

with a command such as "mips_fp_le-gcc -o hello hello.c" and get a
little endian mips binary that runs on your system?  

If so, then you need to modify the .spec file for the given rpm to pick
up the right tool chain, ... and you'll probably need to add macro files
that rpm picks up so  that you can do something like:

rpm -ba --target=xxxxx <spec file>

it works.

That's easier said than done.  I wouldn't know how to do it myself --
someone else has done it for me here.

Pete

---

From: Mike McDonald <[email-masked]>
Date: Fri, 26 Jan 2001 11:47:45 -0800

>Date: Fri, 26 Jan 2001 11:39:17 -0800

>Is that tool chain setup to compile userland apps? Can you cross compile

  Not yet. One of the rpms I'd like to be able to compile is one
of the libc variants. :-)

>If so, then you need to modify the .spec file for the given rpm ...

  I was afraid you were going to say that! I was hoping there was some
way to do it without modifying the spec files by hand. Of course,
the Makefiles would also have to modified to support $(ROOT). Hmm,
maybe I need to write a script that'll build a sandbox that I can
chroot to before I do a 'rpm -ba'.

  Mike McDonald
  [email-masked]

---

From: Florian Lohoff <[email-masked]>
Date: Fri, 26 Jan 2001 21:23:41 +0100

> Another option is native builds, which I personally don't like.

Cross compiling is definitly no option for debian as the dependencies
etc are all made from "ldd binary" which has to fail for cross-compiling.
I guess this also happens to rpm packages so cross-compiling to really
get a correct distribution is definitly no option.

The larger the packages are the harder it is to get them cross-compiled
correctly as they run nifty little check programs from configure which
cant work. I guess you had similar problems as all rpms are
"noarch" which is definitly - ummm - interesting.

I definitly go for native builds - Once you have a working stable 
base you can set up debian autobuilders which will do nearly 
everything for you except signing and uploading the package into
the main repository.

Flo
-- 
Florian Lohoff                  [email-masked]             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?

---

From: Jun Sun <[email-masked]>
Date: Fri, 26 Jan 2001 12:28:38 -0800

> particularly interested in how to specify the tool chain.)

Mike,

Most GNU packages allow you to specify environment variables such
as CC, AR, RANLIB, etc when invoking the "configure" command.  So
you will need to set those variables in your spec file
where "configure" is invoked.

If a package itself is not cross-compiling friendly, you need to 
make it cross-compilable first. :-)

If you only cross-compile one package, I suggest you read through
spec file and do the job manually (untar, applying patch, configure,
make, make install, etc).

Jun

---

From: Jun Sun <[email-masked]>
Date: Fri, 26 Jan 2001 12:51:31 -0800

> get a correct distribution is definitly no option.

There are other ways to figure out the dependency in a cross-compiling 
environment.  We have an internal tool that does just that and more (some
size/fs optimization stuff).  It is not used in the current release, though.

> "noarch" which is definitly - ummm - interesting.

The "noarch" means the installed target is arch-independent.  The
standard setup in mvista CDK is to let target boot from NFS root fs, 
where NFS host can be linux/i386, Linux/ppc and Sun/Sparc (perhaps
Win/i386 as well, I am not sure).  Those packages are meant to be 
installed to all those hosts, and therefore "noarch" :-0.

> everything for you except signing and uploading the package into

Native compiling is easy.  Cross-compiling is cool. :-)

Well, not exactly.  When you are dealing with head-less, disk-less 
memory-scarce embedded devices with ad hoc run-time environments,
cross-compiling is your only choice.

Jun

---

From: Mike McDonald <[email-masked]>
Date: Fri, 26 Jan 2001 13:11:35 -0800

>Date:   Fri, 26 Jan 2001 12:51:31 -0800
>From: Jun Sun <[email-masked]>
>To: Florian Lohoff <[email-masked]>

>installed to all those hosts, and therefore "noarch" :-0.

 Hmm, I would have thought they should be designated for the type of
system they were instead to run on. The fast you're installing them
into an NFS root on some other machine shouldn't change that. Can't
any ole rpm be forced to install on some random NFS server? Then by
your reasoning, all rpms would be noarch, wouldn't they?

>cross-compiling is your only choice.

  Precisely! In our case, we get drops from various contractors who
are doing developement/porting to a wide variety of platforms. (So far
we have i386, mipsel, arm, and sh3. No alpha or sparc yet.) We'll get
multiple drops from the contractors over time. We need to be able to
1) rebuild the binaries from the supplied sources (some vendors have
delivered binaries that did NOT come from the sources they claimed!),
2) build a test suite for that drop and 3) build an initial ramdisk,
bootable CD, or NFS root dir to test the drop. Building the test
environment will include some subset (usually a real small subset) of
the whole drop but we still need to be able to rebuild everything.
Most of these systems we're dealing with have no native compiling
capability, so cross compiling is the only choice. 

  And then sometimes we get tarballs instead of rpms, but that's a
different can of worms.

  Mike McDonald
  [email-masked]

---

From: Mike McDonald <[email-masked]>
Date: Fri, 26 Jan 2001 13:14:03 -0800

>Date:   Fri, 26 Jan 2001 21:23:41 +0100
>From: Florian Lohoff <[email-masked]>
>To: Pete Popov <[email-masked]>

>I definitly go for native builds - 

  If one were to go the native compiling route, what would the minimum
set of rpms needed be? kernel, bin-utils, cc. file-utils? ???

  Mike McDonald
  [email-masked]

---

From: Pete Popov <[email-masked]>
Date: Fri, 26 Jan 2001 16:31:24 -0800

Mike,

> capability, so cross compiling is the only choice.

Here's the recipe for rebuilding all packages from our (MontaVista)
SRPMs:

from ftp.mvista.com:

* download
/pub/CDK/1.2/Latest/MIPS/common/hhl-rpmconfig-0.16-1.noarch.rpm

Install that package. It will go in /opt/hardhat/xxxx

Read carefully /opt/hardhat/config/rpm/README.  That README has all the
info you need to:

* setup your macros files
* download all the tools and SRPMS from the ftp site
* rebuild all of the packages we support, including glibc (glibc is the
only one you have to rebuild as root user)

You can rebuild any package for any of the architectures we support by
doing something like this:

rpm -ba --target=mips_fp_le-linux hhl-glibc.spec

You should be able to setup the environment and start rebuilding
packages within a couple of hours. 

Pete

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Sat, 27 Jan 2001 08:42:34 +0100 (MET)

> get a correct distribution is definitly no option.

 See how my RPM got modified to make use of readelf and objdump if
available to circumvent this problem.  I'm actually going to contribute
these changes to RPM one day (I've just got bored trying to figure the
right e-mail address last time) -- using ldd for this purpose is
definitely broken as it pulls in indirect dependencies (see e.g. dnet vs
non-dnet versions of libX11). 

 All my cross-compiled packages have correct dependencies. 8-}

> everything for you except signing and uploading the package into

 Yep, native builds are more likely to get correct as that's what most
developers out there check (there are actually developers who never heard
of something like a cross-compilation, sigh...).  But not everyone can
afford a week to build glibc or X11... 

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: Karel van Houten <[email-masked]>
Date: Sat, 27 Jan 2001 11:52:05 +0100 (MET)

> set of rpms needed be? kernel, bin-utils, cc. file-utils? ???

It depends on what and how you want to compile. To use rpm, you need
quite a lot tools (db3, patch, sed, grep, find,...). Beside that
you'll at least need glibc, binutils, gcc, and make. But you'll find
out that you'll have to compile flex, bison, m4, automake, autoconf,
and even perl to get rpm builds going. My mipsel native environment
currently has the following packages:

MAKEDEV-3.0.6-5         gcc-c++-2.95.3-14       newt-devel-0.50.17-1
SysVinit-2.78-10        gcc-libstdc++-2.95.3-14 pam-0.72-26
XFree86-4.0.1-1lm       gdbm-1.8.0-5            patch-2.5.4-4
XFree86-devel-4.0.1-1lm gdbm-devel-1.8.0-5      perl-5.6.0-9
XFree86-libs-4.0.1-1lm  gettext-0.10.35-23      popt-1.6-4lm
autoconf-2.13-9         glib-1.2.8-4            pwdb-0.61.1-1
automake-1.4-8          glib-devel-1.2.8-4      python-1.5.2-27
basesystem-7.0-2        glibc-2.1.97-1          python-devel-1.5.2-27
bash-2.04-11            glibc-devel-2.1.97-1    python-tools-1.5.2-27
bc-1.05a-13             gmp-3.0.1-5             readline-4.1-5
bdflush-1.5-14          gmp-devel-3.0.1-5       readline-devel-4.1-5
binutils-2.10.1-3       gpm-1.19.3-4            rpm-4.0-4lm
binutils-libs-2.10.1-3  gpm-devel-1.19.3-4      rpm-build-4.0-4lm
bison-1.28-5            grep-2.4.2-4            sed-3.02-8
byacc-1.9-16            groff-1.16-7            sh-utils-2.0-11
bzip2-1.0.1-3           gtk+-1.2.8-7            slang-1.4.1-5
bzip2-devel-1.0.1-3     gtk+-devel-1.2.8-7      slang-devel-1.4.1-5
cpio-2.4.2-20           gzip-1.3-6              tar-1.13-4
db1-1.85-4              info-4.0-15             tcl-8.3.1-46
db1-devel-1.85-4        ldconfig-1.9.5-16       tcllib-0.4-46
db2-2.4.14-4            libelf-0.7.0-3          termcap-11.0.1-3
db2-devel-2.4.14-4      libelf-devel-0.7.0-3    texinfo-4.0-15
db3-3.1.14-6lm          libpng-1.0.8-1          textutils-2.0e-8
db3-devel-3.1.14-6lm    libpng-devel-1.0.8-1    tix-4.1.0.6-46
db3-utils-3.1.14-6lm    libtermcap-2.0.8-25     tk-8.3.1-46
dev-3.0.6-5             libtermcap-devel-2.0.8- tkinter-1.5.2-27
diffutils-2.7-21        libtool-1.3.4-3lm       unzip-5.41-3
fileutils-4.0x-3        m4-1.4.1-3              utempter-0.5.2-4
findutils-4.1.5-4       make-3.79.1-5           vim-common-5.7-6
flex-2.5.4a-11          ncurses-5.1-2           vim-minimal-5.7-6
gawk-3.0.6-1            ncurses-devel-5.1-2     zlib-1.1.3-12
gcc-2.95.3-14           newt-0.50.17-1          zlib-devel-1.1.3-12

But you surely can start with less... :-)
-- 
Karel van Houten

----------------------------------------------------------
The box said "Requires Windows 95 or better."
I can't understand why it won't work on my Linux computer. 
----------------------------------------------------------

---

From: Mike McDonald <[email-masked]>
Date: Sat, 27 Jan 2001 14:57:24 -0800

>Date: Sat, 27 Jan 2001 11:52:05 +0100 (MET)
>From: Karel van Houten <[email-masked]>
>To: [email-masked] (Mike McDonald)
>
>Mike wrote:
>> 

  I was thinking of what the MINIMUM set of RPMs you needed installed
so you could bootstrap a system up from sources, not what's the
minimum needed to recompile any arbitrary RPM.

>But you surely can start with less... :-)

  With less than 150 files installed in a root file system, I can
install the bin-utils, gcc, make, and glibc RPMs. From there, I should
be able to begin cross compiling the other basic RPMs for a system.
That's my ultimate goal.

  Mike McDonald
  [email-masked]

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 27 Jan 2001 10:50:18 -0800

> afford a week to build glibc or X11... 

Sounds like DECstation results.  Building all the Redhat 7.0 packages which
are on oss + some others which could build for MIPS but don't for some
reason to the point where the build fails takes approx 40h on an Origin 200
with 2 180MHz R10000 processors and 1.5gb RAM.

I recently was told there is some m68k VME system out there which needs
approx. 3 days to rebuild it's kernel.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Sun, 28 Jan 2001 04:10:26 -0800

> minimum needed to recompile any arbitrary RPM.

Really depends on what you want to do.  Many packages detect other packages
or features of other packages.  This builds a big evil network of
dependencies which make bootstrapping somewhat hard.  It's a good idea to
start with an as complete installation as possible.

  Ralf

---

From: Mike McDonald <[email-masked]>
Date: Sun, 28 Jan 2001 09:45:39 -0800

>Date: 	Sun, 28 Jan 2001 04:10:26 -0800
>From: Ralf Baechle <[email-masked]>

  I want to do just the opposite. I want to start with the minimum set
of installed binaries and build a complete binary distribution from
its sources. (That means finding the root of the dependency graph and
starting there, assuming there actually is one. It isn't necessarily a
single rpm. People like to make circular dependancies!)

  Mike McDonald
  [email-masked]

---

From: Geert Uytterhoeven <[email-masked]>
Date: Sun, 28 Jan 2001 19:30:40 +0100 (CET)

> approx. 3 days to rebuild it's kernel.

Since even my 25 MHz 68040 only ca. 6 hours to build a complete 2.4.0 kernel
(incl. lots of modules) these days, that's probably an underclocked 68020 or
so? :-)

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email-masked]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

---

From: Geert Uytterhoeven <[email-masked]>
Date: Sun, 28 Jan 2001 19:27:40 +0100 (CET)

> get a correct distribution is definitly no option.

    [...]

> everything for you except signing and uploading the package into

I really like what they did for ia64 (cfr. the Linux Kongress talk in 1999):
they are running an (emulated) ia64 chrooted environment on an ia32 box.
Binaries in the chrooted environment can be both native (ia32, fast) or
non-native (ia64, emulated and slower). May help for autobuilders for `slow'
architectures (emulated m68k may be faster than the fastest existing '060) as
well...

Using such a technique would allow to let weird stuff like `ldd' and
intermediate created binaries work.

Gr{oetje,eeting}s,

						Geert

P.S. I also think Debian should provide cross-gccs and cross-binutils for all
     architectures it supports, so I don't have to build them myself. And if I
     then could install any lib*-dev*.deb for architecture <arch> into
     /usr/<arch>-linux/ I can cross-compile userspace as well...
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email-masked]

---

From: Pim van Riezen <[email-masked]>
Date: Sun, 28 Jan 2001 20:46:01 +0100

> (incl. lots of modules) these days, that's probably an underclocked 68020 or

I would bet on an 68030 with 4MB of memory and a SCSI Winchester for a
swapdrive. Ah, the memories :).

Pi

-- 
Live phase 1    <-->    RJ45 pin 3      GND     <-->    RJ45 pin 8
Live phase 2    <-->    RJ45 pin 6
Live phase 3    <-->    RJ45 pin 2      Is this suitable?
Neutral         <-->    RJ45 pin 1      Or should we kill phones too?

---

From: Ralf Baechle <[email-masked]>
Date: Sun, 28 Jan 2001 16:02:42 -0800

On Sun, Jan 28, 2001 at 07:30:40PM +0100, Geert Uytterhoeven wrote:

> so?:-)

It's a memory starved '030.

Reminds me of my good old Amiga.  Anybody got me a power supply for it, I
feel like reactiviting it :-)

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Sun, 28 Jan 2001 16:05:18 -0800

> single rpm. People like to make circular dependancies!)

Rpm is a particularly sucky self dependency.  One generation of rpm
inherits certain settings from it's ancestor so bootstrapping only from
sources is a royally sucky.

  Ralf

---

From: Geert Uytterhoeven <[email-masked]>
Date: Mon, 29 Jan 2001 09:44:08 +0100 (MET)

> feel like reactiviting it :-)

What kind of Amiga was it? If it was an A3000, you may get a PSU from Jes,
since he's in 110V country now.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven ------------- Sony Software Development Center Europe (SDCE)
[email-masked] ------------------- Sint-Stevens-Woluwestraat 55
Voice +32-2-7248626 Fax +32-2-7262686 ---------------- B-1130 Brussels, Belgium

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 29 Jan 2001 16:23:36 +0100 (MET)

> single rpm. People like to make circular dependancies!)

 If you have another working Linux system, you may see what I have at
ftp://ftp.ds2.pg.gda.pl/pub/macro/.  I built my mipsel-linux (not complete
yet, e.g. no perl nor X11) system from scratch, i.e. having no MIPS
binaries at all using my i386-linux build system.  All RPM packages have
spec files with explicit "BuildRequires"  dependencies -- you may find
from these what else is needed to build a particular package.

 Only for binutils, gcc and glibc you would need: autoconf, automake,
bash, binutils, bzip2, diffutils, fileutils, findutils, flex, gawk, gcc,
gettext, glibc, grep, gzip, m4, make, patch, perl, rpm, sed, sh-utils,
texinfo, textutils.  You may need additional software to compile some of
these. ;-)

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 29 Jan 2001 16:57:08 +0100 (MET)

> with 2 180MHz R10000 processors and 1.5gb RAM.

 It's mostly RAM-dependent.  A machine with about 4 MB of memory will suck
for compilation regardless of the CPU type.  If you have a decent native
system, why to bother with cross-compiling?

---

From: Ralf Baechle <[email-masked]>
Date: Mon, 29 Jan 2001 13:54:02 -0800

> since he's in 110V country now.

A2000C.

  Ralf

---

From: Mike McDonald <[email-masked]>
Date: Mon, 29 Jan 2001 16:12:04 -0800

>Date:   Mon, 29 Jan 2001 16:57:08 +0100 (MET)
>From: "Maciej W. Rozycki" <[email-masked]>
>To: Ralf Baechle <[email-masked]>

>   If you have a decent native

  Because that's a huge IF! Most of the systems I deal with aren't
"decent" enough to support native compilation of the system. (The
systems of interest to me are embedded and handheld units.)

  Mike McDonald
  [email-masked]

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Tue, 30 Jan 2001 10:46:36 +0100 (MET)

> systems of interest to me are embedded and handheld units.)

 Of course, but Ralf was mentioning some R10k system with 1.5GB of RAM... 
Much enough to buffer all binaries executed during a build as well as all
glibc and X11 sources together with intermediate object and library files
at once. 8-}