commit f7d01f3d036026070c1d6645c9fcfd09cbb0a701 Author: Dustin C. Hatch Date: Mon Jul 25 17:49:44 2022 -0500 Initial commit diff --git a/README.md b/README.md new file mode 100644 index 0000000..2a95d57 --- /dev/null +++ b/README.md @@ -0,0 +1,16 @@ +# Dustin's Kubernetes Cluster + +This repository contains resources for deploying and managing my on-premises +Kubernetes cluster + + +## Cluster Setup + +The cluster primarily consists of libvirt/QEMU+KVM virtual machines. The +Control Plane nodes are VMs, as are the x86_64 worker nodes. Eventually, I +would like to add Raspberry Pi or Pine64 machines as aarch64 nodes. + +All machines run Fedora, using only Fedora builds of the Kubernetes components +(`kubeadm`, `kubectl`, and `kubeadm`). + +See [Cluster Setup](setup/README.md) for details. diff --git a/setup/README.md b/setup/README.md new file mode 100644 index 0000000..2ad377a --- /dev/null +++ b/setup/README.md @@ -0,0 +1,102 @@ +# Cluster Setup + +* Fedora 35 +* Fedora Kubernetes packages 1.22 + + +## Machine Setup + +Add to *pyrocufflink.blue* domain: + +```sh +ansible-playbook \ + -l k8s-amd64-ctrl0.pyrocufflink.blue \ + remount.yml \ + bootstrap.yml \ + pyrocufflink.yml \ + -e ansible_host=172.30.0.167/28 \ + -u root \ + -e @join.creds +``` + +Set up Kubernetes agent (`kubelet`): + +```sh +ansible-playbook -l k8s-amd64-ctrl0.pyrocufflink.blue kubelet.yml -b +``` + + +## Initialize cluster + +Run on *k8s-ctrl0.pyrocufflink.blue*: + +```sh +kubeadm init \ + --control-plane-endpoint kubernetes.pyrocufflink.blue \ + --upload-certs \ + --kubernetes-version=$(rpm -q --qf '%{V}' kubernetes-node) \ + --pod-network-cidr=10.149.0.0/16 +``` + +## Configure Pod Networking + +[Calico] seems to be the best choice, based on its feature completeness, and +a couple of performance benchmarks put it basically at the top. + +```sh +curl -fL\ + -O 'https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml' \ + -O 'https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml' +sed -i 's/192\.168\.0\.0\/16/10.149.0.0\/16/' custom-resources.yaml +kubectl create -f tigera-operator.yaml +kubectl create -f custom-resources.yaml +``` + +Wait for Calico to deploy completely, then restart CoreDNS: + +```sh +kubectl wait -n calico-system --for=condition=ready \ + $(kubectl get pods -n calico-system -l k8s-app=calico-node -o name) +kubectl -n kube-system rollout restart deployment coredns +unset calico_node +``` + + +## Add Worker Nodes + +```sh +kubeadm join kubernetes.pyrocufflink.blue:6443 \ + --token xxxxxx.xxxxxxxxxxxxxxxx \ + --discovery-token-ca-cert-hash sha256:… +``` + + +## Add Control Plane Nodes + +```sh +kubeadm join kubernetes.pyrocufflink.blue:6443 \ + --token xxxxxx.xxxxxxxxxxxxxxxxx \ + --discovery-token-ca-cert-hash sha256:… \ + --control-plane \ + --certificate-key … +``` + +[Calico]: https://projectcalico.docs.tigera.io/getting-started/kubernetes/self-managed-onprem/onpremises + + +## Create Admin user + +```sh +cat < kubeadm-user.yaml < dustin.kubeconfig +```