HPC DIY
We have built, from scratch, a multi-node GPU-enabled computing cluster using Raspberry Pi single-board computers and other commodity hardware (in the spirit of Beowulf-style clusters). We aim for open-source software stack (Linux/Ansible/Slurm/OpenMPI/…). The project has multiple goals:
- to provide a self-hosted GitHub runner for development and testing for our projects which leverage multi-node MPI communication and/or GPU hardware: Numba-MPI, PyMPDATA-MPI and PySDM;
- to provide a playground cluster for learning and teaching parallel computing (MPI + threading and GPUs);
- to build up DevOps know-how on cluster configuration and maintenance;
- to bring a tangible element to our RSE and simulation projects, which can be showcased at open days and outreach events (the cluster mounted on a mobile platform);
- last but not least, to have fun tinkering together with a DIY hardware project!
Compute nodes and interconnection network
As of the present proof-of-concept stage, we have started off with two compute nodes.
Both are off-the-shelf ARM-based NVIDIA Jetson nano boxes (reComputer J1010) with 4GB, quad-core Cortex-A57 CPU and 128-core CUDA GPU each.
The compute nodes and the access node are connected using a Gigabit Ethernet switch.
For debugging and demonstration purposes, we use an HDMI switch and a 7-inch (800x480 pixel, TC-8589556) screen mounted inside the chassis.
Access/storage node
The role of access node is held by a single-plate Raspberry Pi 5 computer with built-in 16GB RAM memory and quad-core ARM 2.4 GHz CPU. Access node has a 16 GB microSD memory card, and is connected via USB to a 2.5” 2TB SSD, which the access node serves to compute nodes using NFS. Raspberry has only one network interface controller, so one network is connected through Ethernet wire and another one has additional USB network adapter. The Raspberry Pi is also connected (through goldpins) to a temperature sensor.
Power supply and consumption monitoring, thermal control, electrical wiring
The external power supply is connected to a Bluetooth power consumption meter (Voltcraft SEM6000).
A 230V fan (Elmeko 10 080 150) is connected to the power strip through a thermostat (Siemens 8MR2171-2BB)
which is factory-set to enable ventilation above 60°C.
A temperature sensor (Joy-it SEN-DHT22 with AM2302 chip) is placed next to the thermostat to enable monitoring of the
temperature.
We use a dedicated grounding busbar (Digitus DN-19 EARTH) connected to the chassis, patch-panels and other metal items.
Chassis and cable management
The system is mounted within a 9-unit 10-inch rack with two shelves (one for access node and SSD; one for compute nodes and the HDMI switch). The 7-inch display is also mounted within the rack (occupying ca. 2.5 units). Access node ports are routed through two 12-port patch-panel with USB, Ethernet and HDMI Keystone modules.
Mobile platform
The rack is attached to a mobile platform which also has a laptop docking station attached for connecting up to two displays used for demonstration purposes (not directly connected to the cluster or the rack).
🎓 Student project opportunities
Stay tuned… (or get in touch with us with your own ideas!)
Contributors from our team
Gracjan Adamus
Michał Wroński
Kamil Kania
Aleksandra Strząbała
Daria Klimaszewska
Agnieszka Żaba
Kacper Derlatka
