In this guest blog post, OpenFaaS founder and Raspberry Pi super-builder Alex Ellis walks us down a five-year-long memory lane explaining how things have changed for cluster users.
I’ve been writing about running Docker on Raspberry Pi for five years now and things have got a lot easier than when I started back in the day. There’s now no need to patch the kernel, use a bespoke OS, or even build Go and Docker from scratch.
My stack of seven Raspberry Pi 2s running Docker Swarm (2016)
Since my first blog post and printed article, I noticed that Raspberry Pi clusters were a hot topic. They’ve only got even hotter as the technology got easier to use and the devices became more powerful.
Back then we used ‘old Swarm‘, which was arguably more like Kubernetes with swappable orchestration and a remote API that could run containers. Load-balancing wasn’t built-in, and so we used Nginx to do that job.
I built out a special demo using kit from Pimoroni.com. Each LED lit up when a HTTP request came in.
Docker load-balanced LED cluster Raspberry Pi
Ask questions and get all the details including the code over on the blog at: http://blog.alexellis.io/iot-docker-cluster/
After that, I adapted the code and added in some IoT sensor boards to create a smart datacenter and was invited to present the demo at Dockercon 2016:
IoT Dockercon Demo
Get all the write-up here: http://blog.alexellis.io/meet-me-at-dockercon/
Docker then released a newer version of Swarm also called ‘Swarm’ and I wrote up these posts:
Docker Swarm mode Deep Dive on Raspberry Pi (scaled)
Please Subscribe to the channel! Get all the details @ http://blog.alexellis.io/live-deep-dive-pi-swarm/
This is still my most popular video on my YouTube channel.
Now that more and more people were trying out Docker on Raspberry Pi (arm), we had to educate them about not running potentially poisoned images from third-parties and how to port software to arm. I created a Git repository (alexellis/docker-arm) to provide a stack of common software.
I wanted to share with users how to use GPIO for accessing hardware and how to create an IoT doorbell. This was one of my first videos on the topic, a live run-through in one take.
Did you know? I used to run blog.alexellis.io on my Raspberry Pi 3
Then we all started trying to run upstream Kubernetes on our 1GB RAM Raspberry Pis with kubeadm. Lucas Käldström did much of the groundwork to port various Kubernetes components and even went as far as to fix some issues in the Go language.
I wrote a recap on everything you needed to know including exec format error and various other things. I also put together a solid set of instructions and workarounds for kubeadm on Raspberry Pi 2/3.
Users often ask what a practical use-case is for a cluster. They excel at running distributed web applications, and OpenFaaS is loved by developers for making it easy to build, deploy, monitor, and scale APIs.
In this post you’ll learn how to deploy a fun Pod to generate ASCII text, from there you can build your own with Python or any other language:
This blog post was one of the ones that got pinned onto the front page of Hacker News for some time, a great feeling when it happens, but something that only comes every now and then.
The instructions for kubeadm and Raspbian were breaking with every other minor release of Kubernetes, so I moved my original gist into a Git repo to accept PRs and to make the content more accessible.
I have to say that this is the one piece of Intellectual Property (IP) I own which has been plagiarised and passed-off the most.
You’ll find dozens of blog posts which are almost identical, even copying my typos. To begin with I found this passing-off of my work frustrating, but now I take it as a vote of confidence.
Shortly after this, Scott Hanselman found my post and we started to collaborate on getting .NET Core to work with OpenFaaS.
This lead to us co-presenting at NDC, London in early 2018. We were practising the demo the night before, and the idea was to use Pimoroni Blinkt! LEDs to show which Raspberry Pi a Pod (workload) was running on. We wanted the Pod to stop showing an animation and to get rescheduled when we pulled a network cable.
It wasn’t working how we expected, and Scott just said “I’ll phone Kelsey”, and Mr Hightower explained to us how to tune the kubelet tolerance flags.
As you can see from the demo, Kelsey’s advice worked out great!
Building a Raspberry Pi Kubernetes Cluster and running .NET Core – Alex Ellis & Scott Hanselman
Join Scott Hanselman and Alex Ellis as they discuss how you can create your own Raspberry Pi cluster that runs Kubernetes on the metal. Then, take it to the …
Fast forward and we’re no longer running Docker, or forcing upstream Kubernetes into 1GB of RAM, but running Rancher’s light-weight k3s in as much as 4GB of RAM.
k3s is a game-changer for small devices, but also runs well on regular PCs and cloud. A server takes just 500MB of RAM and each agent only requires 50MB of RAM due to the optimizations that Darren Shepherd was able to make.
I wrote a new Go CLI called k3sup (‘ketchup’) which made building clusters even easier than it was already and brought back some of the UX of the Docker Swarm CLI.
Kubernetes Homelab with Raspberry Pi 4
Join me for this hands-on tutorial where I build out a Kubernetes Homelab with a Raspberry Pi 4 and get internet access with a LoadBalancer, something normal…
To help combat the issues around the Kubernetes ecosystem and tooling like Helm, which wasn’t available for ARM, I started a new project named arkade . arkade makes it easy to install apps whether they use helm charts or kubectl for installation.
k3s, k3sup, and arkade are all combined in my latest post which includes installing OpenFaaS and the Kubernetes dashboard.
In late March I put together a webinar with Traefik to show off all the OpenFaaS tooling including k3sup and arkade to create a practical demo. The demo showed how to get a public IP for the Raspberry Pi cluster, how to integrate with GitHub webhooks and Postgresql.
The latest and most up-to-date tutorial, with everything set up step by step:
Cloud Native Tools for Developers with Alex Ellis and Alistair Hey
In this Traefik Online Meetup, Alex Ellis, Founder of OpenFaaS, and Alistair Hey, from the OpenFaaS community, will show you how to bootstrap a Kubernetes cl…
In the webinar you’ll find out how to get a public IP for your IngressController using the inlets-operator.
Take-aways
- People will always hate
Some people try to reason about whether you should or should not build a cluster of Raspberry Pis. If you’re asking this question, then don’t do it and don’t ask me to convince you otherwise.
- It doesn’t have to be expensive
You don’t need special equipment, you don’t even need more than one Raspberry Pi, but I would recommend two or three for the best experience.
- Know what to expect
Kubernetes clusters are built to run web servers and APIs, not games like you do with your PC. They don’t magically combine the memory of each node into a single supercomputer, but allow for horizontal scaling, i.e. more replicas of the same thing.
- Not everything will run on it
Some popular software like Istio, Minio, Linkerd, Flux and SealedSecrets do not run on ARM devices because the maintainers are not incentivised to make them do so. It’s not trivial to port software to ARM and then to support that on an ongoing basis. Companies tend to have little interest since paying customers do not tend to use Raspberry Pis. You have to get ready to hear “no”, and sometimes you’ll be lucky enough to hear “not yet” instead.
- Things are always moving and getting better
If you compare my opening statement where we had to rebuild kernels from scratch, and even build binaries for Go, in order to build Docker, we live in a completely different world now. We’ve seen classic swarm, new swarm (swarmkit), Kubernetes, and now k3s become the platform of choice for clustering on the Raspberry Pi. Where will we be in another five years from now? I don’t know, but I suspect things will be better.
- Have fun and learn
In my opinion, the primary reason to build a cluster is to learn and to explore what can be done. As a secondary gain, the skills that you build can be used for work in DevOps/Cloud Native, but if that’s all you want out of it, then fire up a few EC2 VMs on AWS.
Recap on projects
Featured: my 24-node uber cluster, chassis by Bitscope.
- k3sup — build Raspberry Pi clusters with Rancher’s lightweight cut of Kubernetes called k3s
- arkade — install apps to Kubernetes clusters using an easy CLI with flags and built-in Raspberry Pi support
- OpenFaaS — easiest way to deploy web services, APIs, and functions to your cluster; multi-arch (arm + Intel) support is built-in
- inlets — a Cloud Native Tunnel you can use to access your Raspberry Pi or cluster from anywhere; the inlets-operator adds integration into Kubernetes
Want more?
Well, all of that should take you some time to watch, read, and to try out — probably less than five years. I would recommend working in reverse order from the Traefik webinar back or the homelab tutorial which includes a bill of materials.
Become an Insider via GitHub Sponsors to support my work and to receive regular email updates from me each week on Cloud Native, Kubernetes, OSS, and more: github.com/sponsors/alexellis
And you’ll find hundreds of blog posts on Docker, Kubernetes, Go, and more on my blog over at blog.alexellis.io.
Website: LINK
