Several tasks need to be accomplished in order to get the router booting. These are:
- Partition the destination device.
- Mount the appropriate file systems and extract the rootfs tarball.
- Set the admin user password and ssh authorized key(s).
- Install the bootloader to the device.
- Unmount everything and clean up.
We'll use a script to help out with these tasks, and as it's specific to the NG300 hardware, the script will live in board/kerio_ng300/scripts/ng300_deploy.sh.
#!/bin/bash
set -e # Die if anything fails.
DEVICE=$1
if [ -z ${DEVICE} ]
then
echo "usage: ng300_deploy.sh DEVICE"
echo "DEVICE is the device to deploy to, eg: /dev/sda."
exit 1
fi
echo "This will DESTROY all data on ${DEVICE} and install the Nifty Router firmware."
read -p "Are you sure you want to continue [y/N]? " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Yy]$ ]]
then
exit 1
fi
BASE_DIR=$(realpath $(dirname $(readlink -f $0))/../../..)
IMAGE_DIR=${BASE_DIR}/output/images
The initial part of the script checks for the input arguments and prompts for confirmation as this is a destructive operation. It then locates the project base and image output directory. Next the destination device is partitioned and the file systems created:
cat << EOF | sfdisk -q ${DEVICE}
label: dos
start=1M,size=64M,bootable,type=linux
size=512M,type=linux
size=512M,type=linux
type=linux
EOF
p=`ls ${DEVICE}*1 | sed -e "s|${DEVICE}\(.*\)1|\1|"`
mkfs.ext2 -qF ${DEVICE}${p}1 -L boot
mkfs.ext2 -qF ${DEVICE}${p}2 -L root
mkfs.ext2 -qF ${DEVICE}${p}3 -L root
mkfs.ext4 -qF ${DEVICE}${p}4 -L data
Next up is file system extraction. This is a little more complex due to the two root file systems that need to be populated and the separate boot and data partitions. Various options in GNU tar to filter and modify path names make this process easier.
MOUNT=$(mktemp -d)
# Extract the root filesystem(s)
for root in ${DEVICE}${p}2 ${DEVICE}${p}3
do
mount ${root} ${MOUNT}
tar -xf ${IMAGE_DIR}/rootfs.tar --exclude "./boot/*" --exclude "./var/*" -C ${MOUNT}
umount ${MOUNT}
done
#Extract the var filesystem
mount ${DEVICE}${p}4 ${MOUNT}
tar -xf ${IMAGE_DIR}/rootfs.tar --strip-components=2 -C ${MOUNT} ./var
#Set the admin password and authorized keys
MKPASSWD=${BASE_DIR}/output/host/bin/mkpasswd
if [ -z "${ADMIN_PASSWD}" ]; then
ADMIN_PASSWD=`tr -dc 'A-Za-z0-9~!@#$%^&*()=+;:,.<>?|' < /dev/urandom | head -c 16`
fi
ADMIN_ENC_PASSWD="`$MKPASSWD -m sha-256 "$ADMIN_PASSWD"`"
sed -e "s,^admin:[^:]*:,admin:${ADMIN_ENC_PASSWD}:," -i ${MOUNT}/etc/shadow
echo "Admin password: ${ADMIN_PASSWD}"
#set any admin public keys
for key in ${ADMIN_PUBKEY}; do
cat $key >> ${MOUNT}/home/admin/.ssh/authorized_keys
done
umount ${MOUNT}
# Extract the boot filesystem
mount ${DEVICE}${p}1 ${MOUNT}
tar -xf ${IMAGE_DIR}/rootfs.tar --strip-components=2 -C ${MOUNT} ./boot
# Copy the kernel
cp ${IMAGE_DIR}/bzImage ${MOUNT}/linux-current
cp ${IMAGE_DIR}/bzImage ${MOUNT}/linux-prev
# Install grub
${BASE_DIR}/output/host/sbin/grub-install --boot-directory=${MOUNT} --target=i386-pc ${DEVICE}
umount ${MOUNT}
rmdir ${MOUNT}
The admin password is set during the creation of the /var file system on the fourth partition. If the user specifies a plain text password in the environmental variable ADMIN_PASSWD, it is used. Otherwise, a random 16 character password is generated. Any public key files listed in the environmental variable ADMIN_PUBKEY are installed as authorized keys for ssh usage.
Two copies of the kernel are installed to allow booting of a previously known-good system. On upgrade /boot/kernel-current will be copied to /boot/kernel-prev and the root devices updated to match. Speaking of which, we should probably configure GRUB to boot the system. As the bootloader configuration is specific to the board, we'll use another file system overlay to include it. board/kerio_ng300/rootfs_overlay/boot/grub/grub.cfg needs to contain the following:
set default="0"
set timeout="5"
serial --unit=0 --speed=115200
terminal_input serial
terminal_output serial
KERNEL_CMDLINE="rootwait console=ttyS0,115200"
ROOTFS_CURRENT="/dev/sda2"
ROOTFS_PREV="/dev/sda3"
menuentry "Nifty Router" {
linux /linux-current root=${ROOTFS_CURRENT} ${KERNEL_CMDLINE}
}
menuentry "Nifty Router - Previous" {
linux /linux-prev root=${ROOTFS_PREV} ${KERNEL_CMDLINE}
}
Once BR2_ROOTFS_OVERLAY has been updated to include $(BR2_EXTERNAL_NIFTY_ROUTER_PATH)/board/kerio_ng300/rootfs_overlay and a make all issued, we should have a complete image that's ready for deployment.
Testing
While we could deploy the image to a hard drive and boot on the target hardware, it would be easier (and niftier) to test it out with QEMU. Let's create a virtual disk image and deploy to that.
[nifty@monolith nifty_router]$ truncate -s 2G /tmp/router.img
[nifty@monolith nifty_router]$ ls -l /tmp/router.img
-rw-r--r-- 1 nifty nifty 2147483648 Apr 22 12:00 /tmp/router.img
[nifty@monolith nifty_router]$ sudo modprobe loop max_part=15
[nifty@monolith nifty_router]$ sudo losetup -f /tmp/router.img
[nifty@monolith nifty_router]$ sudo ADMIN_PASSWD=test ./board/kerio_ng300/scripts/ng300_deploy.sh /dev/loop0
This will DESTROY all data on /dev/loop0 and install the Nifty Router firmware.
Are you sure you want to continue [y/N]? y
Admin password: test
Installing for i386-pc platform.
Installation finished. No error reported.
[nifty@monolith nifty_router]$
Excellent. It looks like the firmware was successfully deployed to the loop device. Setting the max_part option when loading the loop module allows the loop devices to have partitions, which is very useful in this case. Let's see if we can boot it with QEMU...
[nifty@monolith nifty_router]$ qemu-system-x86_64 -m 2048 -serial stdio -drive id=disk,file=/tmp/router.img,format=raw,if=none -device ahci,id=ahci -device ide-hd,drive=disk,bus=ahci.0 -enable-kvm -display none
As we built the kernel with only SATA/AHCI support, we have to explicitly tell QEMU to create a AHCI device and bind our disk image to it.
What then should have happened was the GRUB menu appearing on the terminal. However, it didn't. What happened instead was a KVM emulation exception. Installing onto a USB drive and booting on a laptop resulted in a boot loop. Something was amiss. I knew that Buildroot 2020.08 worked correctly and that GRUB2 was the same version in each, just with different patches. After some time testing it looked like one of the patches included in commit 09f5832926 was to blame. Instead of sifting through 120 patches and rebuilding to find the culprit(s) I decided it was easier to switch Buildroot to 2020.11.3.
After rebuilding, deploying, and running QEMU again, we get the expected result:
The system successfully loads and boots the kernel and after a little time we're able to log in:
Configuration Tweaks
The system boots and is useable, which is fantastic. However, there's a few things that need to be taken care of. Firstly, there is a message in the console output above:
No persistent location to store SSH host keys. New keys will be
generated at each boot. Are you sure this is what you want to do?
Starting dropbear sshd: OK
This comes from the dropbear init script and is caused by /etc/dropbear ultimately pointing to /run/dropbear, which is a temporary filesystem. We can fix this with a few modifications to the filesystem overlay:
[nifty@monolith nifty_router]$ mkdir rootfs_overlay/etc
[nifty@monolith nifty_router]$ ln -s /var/etc/dropbear rootfs_overlay/etc/dropbear
[nifty@monolith nifty_router]$ mkdir -p rootfs_overlay/var/etc/dropbear
[nifty@monolith nifty_router]$ touch rootfs_overlay/var/etc/dropbear/.empty
Next up, we should set fstrim to be called periodically to discard unused blocks in /var. We can configure a crontab for root to achieve this in rootfs_overlay/var/spool/cron/crontabs/root:
# min hour day month weekday command
* * * * 0 /sbin/fstrim -a
* 0 * * * /usr/sbin/logrotate /etc/logrotate.conf
All mounted filesystems will be trimmed once a week. Additionally, the system logs will get rotated daily. However, an init script isn't provided for cron by default. We can provide our own in rootfs_overlay/etc/init.d/S25crond:
#!/bin/sh
#
# Starts crond.
#
start() {
printf "Starting crond: "
umask 077
start-stop-daemon -S -q -p /var/run/crond.pid \
--exec /usr/sbin/crond -- -l 6
[ $? = 0 ] && echo "OK" || echo "FAIL"
}
stop() {
printf "Stopping crond: "
start-stop-daemon -K -q -p /var/run/crond.pid
[ $? = 0 ] && echo "OK" || echo "FAIL"
}
restart() {
stop
start
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload)
restart
;;
*)
echo "Usage: $0 {start|stop|restart}"
exit 1
esac
exit $?
Finally, a nicer shell prompt would be appreciated. rootfs_overlay/etc/profile.d/prompt.sh just needs a single line:
export PS1='[\u@\h \W]\$ '
Running make all again should quickly include the changes that were made above. After redeploying, the system can retain it's SSH keys, trim the filesystems, and it looks a bit nicer to use.
The next article will cover Networking and Firewall Setup, which has been the whole point of the exercise.