Hey! I have been using Ansible to deploy Dockers for a few services on my Raspberry Pi for a while now and it’s working great, but I want to learn MOAR and I need help…
Recently, I’ve been considering migrating to bare metal K3S for a few reasons:
- To learn and actually practice K8S.
- To have redundancy and to try HA.
- My RPi are all already running on MicroOS, so it kind of make sense to me to try other SUSE stuff (?)
- Maybe eventually being able to manage my two separated servers locations with a neat k3s + Tailscale setup!
Here is my problem: I don’t understand how things are supposed to be done. All the examples I find feel wrong. More specifically:
- Am I really supposed to have a collection of small yaml files for everything, that I use with
kubectl apply -f?? It feels wrong and way too “by hand”! Is there a more scripted way to do it? Should I stay with everything in Ansible ?? - I see little to no example on how to deploy the service containers I want (pihole, navidrome, etc.) to a cluster, unlike docker-compose examples that can be found everywhere. Am I looking for the wrong thing?
- Even official doc seems broken. Am I really supposed to run many helm commands (some of them how just fails) and try and get ssl certs just to have Rancher and its dashboard ?!
I feel that having a K3S + Traefik + Longhorn + Rancher on MicroOS should be straightforward, but it’s really not.
It’s very much a noob question, but I really want to understand what I am doing wrong. I’m really looking for advice and especially configuration examples that I could try to copy, use and modify!
Thanks in advance,
Cheers!
And this is why I do not like K8s at all. The only reason to use it is to have something on your CV. Besides that, Docker Swarm and Hashicorp Nomad feel a lot better and are a lot easier to manage.
I personally feel like K8s has a purpose but not in homelab since our infrastructure is usually small. I don’t need clever load-balancing or autoscaling for most of my work.
Of course it is overkill for a homelab. The other features you mentioned, can be achieved by Nomad or Swarm as well. And with Nomad you don’t even have to use the Docker engine.
Just ask yourself the following question: why is helm so popular? Why do I need a third party scripting language just for K8s?
You clearly will feel that K8s did many things right. 10 years ago. But we learned from that. And operations cost are exploding everywhere I see K8s in use (with or without Helm). Weird side effects, because at this layer you almost have an indefinite amount of edge cases.
That’s why I move away from K8s. To make very large and complex platforms manageable for a small operations team. The DevOps Engineers don’t like that obviously, because it is a major skill on the job market. In the end, I have to prioritize and all I can do is spread awareness, that K8s was great at some point, as was Windows 98 SE.