infra
/
kubernetes
1
0
Fork 0
Code Pull Requests 4 Activity
Resources for deploying and managing my personal Kubernetes cluster
16 Commits
9 Branches
0 Tags
3.5 MiB
Shell 83.2%
JavaScript 14.8%
Awk 2%
 
 
 
Go to file
Dustin 6c7dcce90b setup: switch back to ext4 on lvm 
Originally, I decided to use *btrfs* subvolumes to create writable
directories inside otherwise immutable locations, such as for
`/etc/cni/net.d`, etc.  I figured this would be cleaner than
bind-mounting directories from `/var`, and would avoid the trouble of
determining an appropriate volume sizes necessary to make them each
their own filesystem.

Unfortunately, it turns out that *cri-o* may still have some issues with
its *btrfs* storage driver.  One [blog post][0] hints at performance
issues in *containerd*, and it seems they may apply to *cri-o* as well.
I certainly encountered performance issues when attempting to run `npm`
in a Jenkins job running in a Kubernetes pod.  There is definitely a
[performance issue with `npm`][1] when running in a container, which may
or may not have been exacerbated by the *btrfs* storage driver.

In any case, upstream [does not reecommend][2] using the *btrfs* driver,
performance notwithstanding.  The *overlay* driver is much more widely
used and tested.  Plus, it's easier to filter out container layers from
filesystem usage statistics simply by ignoring *overlay* filesystems.

[0]: https://blog.cubieserver.de/2022/dont-use-containerd-with-the-btrfs-snapshotter/
[1]: https://github.com/npm/cli/issues/3208#issuecomment-1002990902
[2]: https://github.com/containers/storage/issues/929
 		2022-07-31 17:09:03 -05:00
docker-distribution docker-distribution: Deploy OCI image registry 2022-07-31 01:15:01 -05:00
ingress ingress: Add manifest for ingress-nginx 2022-07-31 00:57:12 -05:00
jenkins jenkins: Fix typo in README 2022-07-31 00:42:42 -05:00
phpipam phpipam: Add manifest for phpipam 2022-07-31 01:31:53 -05:00
setup setup: switch back to ext4 on lvm 2022-07-31 17:09:03 -05:00
storage storage: Add manifest for Longhorn 2022-07-31 00:57:53 -05:00
README.md README: Add storage section 2022-07-31 01:38:46 -05:00

README.md

Dustin's Kubernetes Cluster

This repository contains resources for deploying and managing my on-premises Kubernetes cluster

Cluster Setup

The cluster primarily consists of libvirt/QEMU+KVM virtual machines. The Control Plane nodes are VMs, as are the x86_64 worker nodes. Eventually, I would like to add Raspberry Pi or Pine64 machines as aarch64 nodes.

All machines run Fedora, using only Fedora builds of the Kubernetes components (kubeadm, kubectl, and kubeadm).

See Cluster Setup for details.

Jenkins Agents

One of the main use cases for the Kubernetes cluster is to provide dynamic agents for Jenkins. Using the Kubernetes Plugin, Jenkins will automatically launch worker nodes as Kubernetes pods.

See Jenkins Kubernetes Integration for details.

Persistent Storage

Persistent storage for pods is provided by Longhorn. Longhorn runs within the cluster and provisions storage on worker nodes to make available to pods over iSCSI.

See Persistent Storage Using Longorn for details.