Motivation
I recently stumbled upon HypriotOS while looking for Docker-ready distributions for my Raspberry Pi 3B+. I flashed this onto and SD card and started playing around with it. It works incredibly well, but I noticed that it was built for armv7l which is a 32-bit implementation. Since the Raspberry Pi 3B+ has a 4x core Cortex-A53 which is 64 bit, I wanted to make use of the 64 bit processor! I’ve worked with Yocto before (in fact, my day job uses Yocto), so I decided I’d build my own. For more on containers, see my other blog posts on what it is, what it can be used for, and how to use it.
Process
Prerequisites
Yocto is a build, not a distribution, so it is quite resource intensive. I would suggest having at least 60GB free disk space, and a quad core machine at a minimum. You’ll also need a Linux installation, and the build essentials installed (git, gcc, etc.).
EDIT: Checking the disk usage after the builds shows around 55GB used. On a 2C4T Intel Core i5 it took about 2H to build.
Project Setup
First, create a working directory and clone all the piece-parts we need to build.
mkdir rpi3bplus-build-yocto
cd rpi3bplus-build-yocto
git clone git://git.yoctoproject.org/poky
git clone git://git.openembedded.org/meta-openembedded
git clone git://git.yoctoproject.org/meta-raspberrypi
git clone git://git.yoctoproject.org/meta-virtualization
Next create the project and add the base layers.
source poky/oe-init-build-env rpi64
bitbake-layers add-layer ../meta-raspberrypi
bitbake-layers add-layer ../meta-openembedded/meta-oe
bitbake-layers add-layer ../meta-openembedded/meta-python
bitbake-layers add-layer ../meta-openembedded/meta-perl
bitbake-layers add-layer ../meta-openembedded/meta-networking
bitbake-layers add-layer ../meta-openembedded/meta-filesystems
bitbake-layers add-layer ../meta-virtualization
Edit the conf/local.conf file sections as needed:
MACHINE ??= "raspberrypi3-64"
CORE_IMAGE_EXTRA_INSTALL += "kernel-modules htop openssh iperf3 docker-ce bash ntp "
INHERIT += "extrausers"
EXTRA_USERS_PARAMS += " useradd pi; \
usermod -p 'raspberry' pi; \
usermod -a -G sudo pi; \
usermod -P root root; "
DISTRO_FEATURES_append = " virtualization"
Next we will build the image. This will take a while depending on how beefy your build machine is.
bitbake core-image-minimal
Parsing recipes: 100% |########################################################################################################################################################################################################| Time: 0:02:16
Parsing of 2540 .bb files complete (0 cached, 2540 parsed). 3827 targets, 140 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "raspberrypi3-64"
DISTRO = "poky"
DISTRO_VERSION = "3.0"
TUNE_FEATURES = "aarch64 cortexa53 crc"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "master:6bb4a252199cfd5d44ad7ab6fc4118c80a1aae92"
meta-raspberrypi = "master:a0a5d3848e76b7f90ef8e42c56211da78db1c7ba"
meta-oe
meta-python
meta-perl
meta-networking
meta-filesystems = "master:d9f3e6dbed8e5d96f2069280f0a566af89afb2fa"
meta-virtualization = "master:5fb77ae4c4e1015e40257f9e59e16c497e30c53c"
After a couple of hours, you will have a completed, ready to flash SD Image at <build>/rpi3bplus-build-yocto/rpi64/tmp/deploy/images/raspberrypi3-64/core-image-minimal-raspberrypi3-64.rpi-sdimg which you can burn to an SD card with:
dd if=./core-image-minimal-raspberrypi3-64.rpi-sdimg of=/dev/sdX status=progress
Your image should boot up and docker will be running!
NOTE: You will manually have to set the date to pull from Docker registries over HTTPS because the certification validation will fail otherwise. We could add and configure NTP to fix the but… for another day. Use:
date "+%d-%m-%C%y %H:%M:%S" -s "2019-12-22 00:30:01"
There is one last thing you’ll want to do. You’ll want to use GParted or similar tool to enlarge your filesystem partition to take up the rest of your SD card so you will have some room for container images. You need to do this because the default SD Image builder script only provides enough space for the root filesystem and no more. In my configuration it ended up being ~300MB, so I expanded it take up the remaining ~59GB. See example below:
Results
As it turns out, I have a system that can build and run 64-bit Docker images and is pretty darn slim. Here’s a screenshot of htop running.. only 12 processes – not bad!
Some Metrics
The is the minimal image and it has 12 processes, with an incredibly low footprint. I loaded up an Ubuntu container and installed iperf3, then ran a test to my laptop. I was impressed to find the following result:
[ ID] Interval Transfer Bandwidth Retr
[ 5] 0.00-10.13 sec 116 MBytes 96.0 Mbits/sec 9 sender
[ 5] 0.00-10.13 sec 113 MBytes 93.8 Mbits/sec receiver
Even running that test, the load on the Pi was:
root@raspberrypi3-64:~# cat /proc/loadavg
0.05 0.22 0.19 1/141 2125
root@raspberrypi3-64:~#
Downloads
Clear Signed SHA256 Sum of the images below (look up my public key 0x2408A81C)
