Raspberry Pi 3 toolchain on CentOS 7

I heard a lot about Raspberry Pi boards but until now I had no need nor time to work with one.

However, recently I purchased a Dodge Journey R/T and found that although I love the car I am so disappointed with its software and hard-wired logic that I decided to experiment a bit and fix the most annoying things.

Since almost everything inside the car is talking over the CAN bus I needed some kind of a enclave inside the car where I could run my code and inject/intercept CAN messages.

I looked around and found that I can build the desired appliance using Raspberry Pi 3 (Model B) + PiCAN 2 HAT board.

Once the hardware was delivered to my home the time came to start building the software side of things. My distribution of choice for this project became CentOS 7 (userland), however, building stuff on the Raspberry Pi itself was a painful and long process, so I needed a proper toolchain to be able to utilise much more powerful hardware and do builds quicker.

The following is a session dump (with some notes) on how I built my toolchain on an AWS EC2 instance which was running a minimal CentOS 7 as its OS.

I spawned an EC2 instance (you need at least 8GB of free space there to be able to build the toolchain), logged in, and ensured that my system is up to date (and update it if it was not):

[root@localhost ~]# yum -y update
Loaded plugins: at-exit, fastestmirror, post-transaction-actions
Loading mirror speeds from cached hostfile
 * base: mirror.domain.tld
 * extras: mirror.domain.tld
 * updates: mirror.domain.tld
No packages marked for update

The easiest way to build a toolchain is to use the crosstool-ng project, so I went ahead and downloaded the latest version with the corresponding GPG signature, verified the signature (although the verification was very superficial since Bryan Hundven is not in my Web of Trust and the crosstool-ng website was not providing an HTTPS page to confirm the fingerprint of the key), and unpacked the sources:

[root@localhost ~]# yum -y install bison bzip2 flex gcc gcc-c++ glibc-static gperf help2man libstdc++-static libtool make ncurses-devel texinfo wget
[root@localhost ~]# useradd -m build
[root@localhost ~]# su - build
[build@localhost ~]$ wget -q 'http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.xz'
[build@localhost ~]$ wget -q 'http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.xz.sig'
[build@localhost ~]$ gpg --recv-keys 35B871D1 --keyserver pgp.surfnet.nl
gpg: "--keyserver" not a key ID: skipping
gpg: "pgp.surfnet.nl" not a key ID: skipping
gpg: requesting key 35B871D1 from hkp server keys.gnupg.net
gpg: /home/build/.gnupg/trustdb.gpg: trustdb created
gpg: key 35B871D1: public key "Bryan Hundven " imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
[build@localhost ~]$ gpg --verify crosstool-ng-1.22.0.tar.xz.sig
gpg: Signature made Fri Nov 20 13:09:26 2015 UTC using RSA key ID 35B871D1
gpg: Good signature from "Bryan Hundven "
gpg:                 aka "Bryan Hundven "
gpg:                 aka "Bryan Hundven "
gpg:                 aka "Bryan Hundven "
gpg:                 aka "[jpeg image of size 3080]"
gpg:                 aka "[jpeg image of size 16246]"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 561E D9B6 2095 88ED 23C6  8329 CAD7 C8FC 35B8 71D1
[build@localhost ~]$ tar xJSf crosstool-ng-1.22.0.tar.xz
[build@localhost ~]$ rm crosstool-ng-1.22.0.tar.xz*

The next step was to prepare the environment for the crosstool-ng build process:

[build@localhost ~]$ cd crosstool-ng
[build@localhost ~]$ /configure --prefix=$HOME/ct-ng
checking build system type... x86_64-pc-linux-gnu
[skipped]
configure: creating ./config.status
config.status: creating Makefile
[build@localhost ~]$ make
  SED    'ct-ng'
[skipped]
  SED    'docs/ct-ng.1'
  GZIP   'docs/ct-ng.1.gz'
[build@localhost ~]$ make install
  GEN    'config/configure.in'
[skipped]

For auto-completion, do not forget to install ct-ng.comp into your bash completion directory (usually /etc/bash_completion.d/)

[build@localhost ~]$ export PATH=$HOME/ct-ng/bin:$PATH
[build@localhost ~]$ cd

Once the above was done we are ready to actually configure and build our toolchain:

[build@localhost ~]$ mkdir ~/builddir
[build@localhost ~]$ cd ~/builddir
[build@localhost ~]$ ct-ng menuconfig

I was quite happy with the defaults and the only changes I made were the following:

• Paths and misc options -> Try features marked as EXPERIMENTAL -> check

  You may skip this but I prefer to see all available options when I go through the list of configurable parameters, hence I enabled this;

• Target options -> Target Architecture -> arm

  My board is Raspberry Pi 3 (Model B) which runs on ARMv8 Cortex-A53, yours may be different but most likely it is also ARM based :)

• Target options -> Bitness -> 64-bit

  There is actually almost no point in building a 64-bit version when you are running on such a limited device as Raspberry, but at the time I was writing this blog post I was building the AArch64 version of the toolchain. I would recommend to stick with the 32-bit version of the toolchain if you do not have any specific requirements for the 64-bit one;

• Target options -> Architecture level -> armv8-a

  Since my only target was Raspberry Pi 3 (Model B) and the toolchain was created for the personal use I did additional optimisation for my board.

  If you are not sure or if you want a toolchain that would produce more compatible code for a range of ARM devices, then leave this setting at its default, i.e. blank

• Target options -> Emit assembly for CPU -> cortex-a53

  Actually, this supersedes the previous option since it is more explicit. If you are not optimising for a specific hardware, then leave it at the default setting (which is blank).

  Also, there are more optimisations, e.g. you can specify that FPU for this board is neon-fp-armv8, etc. but to keep it simple I decided not to showcase that.

• Toolchain options -> Tuple’s vendor string -> rpi3

  This is optional but I like my toolchain to be as descriptive as possible, e.g. with this setting binaries will be prefixed with aarch64-rpi3-linux-gnueabi-

• Operating System -> Target OS -> linux

  It is essential to set this to “linux” otherwise crosstool-ng will assume that you are building a toolchain for a bare-bone hardware and the produced binaries would not be properly linked to work with the OS on the board

Once we are comfortable with the configuration of the future toolchain we exit the configuration menu (saving the changes) and launch the build process (it would take approximately an hour on AWS t2.small instance):

configuration written to .config

*** End of the configuration.
*** Execute 'ct-ng build' to start the build or try 'ct-ng help'.

[build@localhost ~]$ ct-ng build
[INFO ]  Performing some trivial sanity checks
[INFO ]  Build started 20160522.070915
[skipped]
[INFO ]  Build completed at 20160522.080629
[INFO ]  (elapsed: 57:14.18)
[INFO ]  Finishing installation (may take a few seconds)...
[57:15] /

Seems that our Raspberry Pi 3 toolchain is ready to be used. Let’s compile something and see how it goes:

[build@localhost ~]$ export PATH=$HOME/x-tools/aarch64-rpi3-linux-gnueabi/bin:$PATH
[build@localhost ~]$ aarch64-rpi3-linux-gnueabi-gcc --version
aarch64-rpi3-linux-gnueabi-gcc (crosstool-NG crosstool-ng-1.22.0) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[build@localhost ~]$ printf '#include \nint main(){printf("Hello\\n"); return 0;}\n' > sample.c
[build@localhost ~]$ aarch64-rpi3-linux-gnueabi-gcc -o sample sample.c
[build@localhost ~]$ ls -l sample
-rwxr-xr-x. 1 build build 7672 May 22 08:19 sample
[build@localhost ~]$ file sample
sample: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 4.3.0, not stripped

Well, it is time to build some awesome stuff for Raspberry Pi 3, I guess, but this will be in another post. :)