I have recently started using OpenShift for work and thought, why not create my own cluster at home? I have a bunch of Raspberry Pis (and Orange Pi) lying around, and it would be nice if I can have a playground cluster available.
After some digging, I have decided to use K3s as it seems more appropriate for IoT. I am using the latest Raspberry Pi OS image (2020-08-20-raspios-buster-armhf-lite.img) available. Installation went smoothly. I was able to run pods, have the pods communicate via services etc. However, when attempting to create an ingress, it fails.
My deployment and service configuration:
apiVersion: apps/v1 kind: Deployment apiVersion: apps/v1 kind: Deployment metadata: name: mysite labels: app: mysite spec: replicas: 1 selector: matchLabels: app: mysite template: metadata: labels: app: mysite spec: containers: - name: nginx image: httpd:latest ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: mysite-service spec: selector: app: mysite ports: - protocol: TCP port: 80
My ingress configuration:
apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: mysite-ingress annotations: kubernetes.io/ingress.class: "traefik" spec: rules: - host: k8s.example.com http: paths: - backend: serviceName: mysite-service servicePort: 80
As you can see, everything should be working fine, but it's not. The ingress was created, but it's acting as if it's not.
After days of digging, I was finally able to figure out the problem. As it turns out, Traefik, the default ingress controller of K3s, doesn't work with iptables-nft (for now). This can easily be resolved by switching iptables to legacy and rebooting:
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy sudo reboot -h now
Once booted up, you should be able to access your pod via the ingress.