{"id":436,"date":"2020-02-14T23:27:56","date_gmt":"2020-02-14T15:27:56","guid":{"rendered":"https:\/\/codestrian.com\/?p=436"},"modified":"2020-02-14T23:36:17","modified_gmt":"2020-02-14T15:36:17","slug":"setting-up-a-pi-cluster-with-netboot","status":"publish","type":"post","link":"https:\/\/codestrian.com\/index.php\/2020\/02\/14\/setting-up-a-pi-cluster-with-netboot\/","title":{"rendered":"Setting up a Pi Cluster with Netboot"},"content":{"rendered":"

\"\"
\nThe very first version of raspberry pi was introduced back in 2012. Since then, it has gone through 3 iterations of improvements. Over the years, it has grown much in popularity to be used as a cheap computing device for educational purpose.
\n\"\"
\nMy very first raspberry pi was the version 1 model B which only had 2 USB, 512MB of RAM and a ARM CPU that is running below 1GHz. I used that machine to learn about hosting my own servers. And over the past few years, I have always wanted to set up my own Pi cluster for learning purpose (I studied parallel computing as my specialization back in school). Towards the end of last year, I have decided to purchase 4 of the latest raspberry pi 4 model B to create the cluster that I have always wanted to build. <\/p>\n

Things I bought<\/h1>\n

Raspberry Pi 4 Model B (4GB)<\/a> X 4
\nAnker USB Charger<\/a> X 1
\nC4Lab Bramble Case<\/a> X 1
\nDlink 5 Ports POE Switch<\/a> X 1
\n0.5m Cat6 Ethernet Cable<\/a> X 5
\nUSB to USB-C cable<\/a> X 4
\nSandisk Extreme 32GB MicroSD<\/a> X 4<\/p>\n

Note:<\/strong> Although I have bought 4 microSD, I only used 1 to set up netboot on raspberry pi. I will still recommend getting 4 in the event that you prefer to boot from the microSD.<\/p>\n

First of all, download the latest version of raspbian lite image from raspberry pi website. And burn that image to the microSD card. If you are using Linux, you may use DD command. If you are using windows, you may use rufus.<\/p>\n

Next, we will need to update the bootloader on raspberry pi to enable network boot.<\/p>\n

sudo apt-get update\nsudo apt-get upgrade\nwget https:\/\/github.com\/raspberrypi\/rpi-eeprom\/raw\/master\/firmware\/beta\/pieeprom-2020-01-17.bin\nrpi-eeprom-config pieeprom-2020-01-17.bin > bootconf.txt<\/code><\/pre>\n
Next, we will need to edit the bootloader config instructing it to attempt netboot before booting from SD card.
\nFor the list of available bootloader options, you may refer it here<\/a><\/p>\n
sed -i s\/0x1\/0x21\/g bootconf.txt <\/code><\/pre>\n
Now we need to create a new copy of the bootloader firmware with the new boot config.<\/p>\n
rpi-eeprom-config --out pieeprom-2020-01-17-netboot.bin --config bootconf.txt pieeprom-2020-01-17.bin\nsudo rpi-eeprom-update -d -f .\/pieeprom-2020-01-17-netboot.bin\ncat \/proc\/cpuinfo<\/code><\/pre>\n
Next take note of the last 8 digits of Serial number. e.g. 07c38cdc<\/em>
\nWe will need it later for netboot configuration<\/p>\n
Hardware    : BCM2835\nRevision    : c03112\nSerial      : 1000000007c38cdc<\/code><\/pre>\n
That is pretty much all we have to do on the raspberry pi.<\/p>\n
Set up the NFS and TFTP<\/h1>\n
Now we need to set up a server which allows the raspberry pi to boot from.
\nFor me, I have opted to use ubuntu.<\/p>\n
Before we begin, let's install the tools we need.<\/p>\n
sudo apt install unzip kpartx dnsmasq nfs-kernel-server<\/code><\/pre>\n
First, we will need to download the raspbian image and unzip it. Since we are gonna run it in headless mode, we will download the lite version which comes without GUI.<\/p>\n
wget http:\/\/downloads.raspberrypi.org\/raspbian_lite\/images\/raspbian_lite-2020-02-07\/2020-02-05-raspbian-buster-lite.zip\nunzip 2020-02-05-raspbian-buster-lite.zip<\/code><\/pre>\n
Next we will mount the image and map the 2 partitions to the 2 directories, root<\/strong> and boot<\/strong>.<\/p>\n
sudo kpartx -a -v 2020-02-05-raspbian-buster-lite.img\nmkdir root\nmkdir boot\nsudo mount \/dev\/mapper\/loop0p1 boot\/\nsudo mount \/dev\/mapper\/loop0p2 root\/<\/code><\/pre>\n
Next we will copy all the files in boot<\/strong> and root<\/strong> to a directory which will be mounted as NFS directory.<\/p>\n
sudo mkdir -p \/rpi\/1\nsudo cp -a root\/* \/rpi\/1\/\nsudo cp -a boot\/* \/rpi\/1\/boot\/<\/code><\/pre>\n
After the step above, we will need to replace 2 files in the boot folder.<\/p>\n
cd \/rpi\/1\/boot\nsudo rm start4.elf\nsudo rm fixup4.dat\nsudo wget https:\/\/github.com\/Hexxeh\/rpi-firmware\/raw\/master\/start4.elf\nsudo wget https:\/\/github.com\/Hexxeh\/rpi-firmware\/raw\/master\/fixup4.dat<\/code><\/pre>\n
In order to serve the boot partition across the network, we will need to mount it. Still remember the serial number that we have recorded? We will use it here as the partition name. This is important as raspberry pi will look for this folder on the server.<\/p>\n
sudo mkdir -p \/tftpboot\/07c38cdc\necho "\/rpi\/4\/boot \/tftpboot\/07c38cdc none defaults,bind 0 0" | sudo tee -a \/etc\/fstab\nsudo mount \/tftpboot\/07c38cdc\nsudo chmod 777 \/tftpboot\nsudo touch \/rpi\/1\/boot\/ssh\nsudo sed -i \/UUID\/d \/rpi\/3\/etc\/fstab<\/code><\/pre>\n
After the pi has loaded the boot partion, we need to let it know where the root partition is. To do that, we need to edit the cmdline.txt<\/strong> in the boot partition. (Replace 192.168.3.120 with your server IP)<\/p>\n
echo "console=serial0,115200 console=tty root=\/dev\/nfs nfsroot=192.168.3.120:\/rpi\/4,vers=3 rw ip=dhcp rootwait elevator=deadline" | sudo tee \/rpi\/1\/boot\/cmdline.txt<\/code><\/pre>\n
After all the above is done, we just need to enable \/rpi\/1<\/em> as NFS directory and enable the relevant services so that raspberry pi can locate the server and NFS directory. (Replace 192.168.3.255 with your network mask)<\/p>\n
echo "\/rpi\/1 *(rw,sync,no_subtree_check,no_root_squash)" | sudo tee -a \/etc\/exports\nsudo systemctl enable rpcbind\nsudo systemctl enable nfs-kernel-server\nsudo systemctl restart rpcbind\nsudo systemctl restart nfs-kernel-server\n\necho 'dhcp-range=192.168.3.255,proxy' | sudo tee -a \/etc\/dnsmasq.conf\necho 'log-dhcp' | sudo tee -a \/etc\/dnsmasq.conf\necho 'enable-tftp' | sudo tee -a \/etc\/dnsmasq.conf\necho 'tftp-root=\/tftpboot' | sudo tee -a \/etc\/dnsmasq.conf\necho 'pxe-service=0,"Raspberry Pi Boot"' | sudo tee -a \/etc\/dnsmasq.conf\nsudo systemctl enable dnsmasq\nsudo systemctl restart dnsmasq<\/code><\/pre>\n
Repeat the above steps for your other raspberry pi and you will have a cluster.<\/p>\n
If you have done everything correctly, you should see the log appearing in \/var\/log\/syslog<\/em>.<\/p>\n","protected":false},"excerpt":{"rendered":"
The very first version of raspberry pi was introduced back in 2012. Since then, it has gone through 3 iterations of improvements. Over the years, it has grown much in popularity to be used as a cheap computing device for educational purpose. My very first raspberry pi was the version […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,42,53],"tags":[],"class_list":["post-436","post","type-post","status-publish","format-standard","hentry","category-hardware","category-linux","category-raspberry"],"_links":{"self":[{"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/posts\/436","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/comments?post=436"}],"version-history":[{"count":14,"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/posts\/436\/revisions"}],"predecessor-version":[{"id":453,"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/posts\/436\/revisions\/453"}],"wp:attachment":[{"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/media?parent=436"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/categories?post=436"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codestrian.com\/index.php\/wp-json\/wp\/v2\/tags?post=436"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}