Harry Tang

How to install & configure Longhorn for Kubernetes

How Longhorn Works
How Longhorn Works

Introduction

Longhorn is a free, open-source, lightweight, reliable, and easy-to-use distributed block storage system for Kubernetes cluster. This guide will show you how to install and configure Longhorn for your k8s cluster.

Prerequisites

  • A k8s cluster >=v1.25.
  • Helm v3.
  • Basic k8s knowledge.

Step-by-step Guide

  1. Create the longhorn-system namespace and configure the Helm repository.

    kubectl create ns longhorn-system
namespace/longhorn-system created

helm repo add longhorn https://charts.longhorn.io
"longhorn" has been added to your repositories

helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "longhorn" chart repository
Update Complete. ⎈Happy Helming!⎈

  2. Create the longhorn.values.yaml for storing the Longhorn configuration values:

    # https://artifacthub.io/packages/helm/longhorn/longhorn?modal=values

  3. The first setting we want to adjust is the Data Locality; set it to best-effort to have Longhorn try to keep a replica on the same node as the attached volume:

    persistence:
 defaultDataLocality: 'best-effort'

  4. If you are using microk8s, you will also need to set the kubeletRootDir point to your kubelet directory:

    csi:
  kubeletRootDir: /var/snap/microk8s/common/var/lib/kubelet

  5. The final values should look like below:

    # https://artifacthub.io/packages/helm/longhorn/longhorn?modal=values    
persistence:
  defaultDataLocality: 'best-effort'
csi:
  kubeletRootDir: /var/snap/microk8s/common/var/lib/kubelet

  6. Now it is time to install the Longhorn system:

    helm install longhorn longhorn/longhorn \
  --namespace longhorn-system \
  --values longhorn.values.yaml
NAME: longhorn
LAST DEPLOYED: Wed Jul 24 07:16:08 2024
NAMESPACE: longhorn-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Longhorn is now installed on the cluster!

Please wait a few minutes for other Longhorn components such as CSI deployments, Engine Images, and Instance Managers to be initialized.

Visit our documentation at https://longhorn.io/docs/

  7. Check the status of the installation:

    kubectl get pods -n longhorn-system
NAME                                                READY   STATUS    RESTARTS   AGE
csi-attacher-5f8b9cbbdc-2dbdg                       1/1     Running   0          45s
csi-attacher-5f8b9cbbdc-hq7ps                       1/1     Running   0          45s
csi-attacher-5f8b9cbbdc-k2tpq                       1/1     Running   0          45s
csi-provisioner-6d85c78995-b4b4r                    1/1     Running   0          45s
csi-provisioner-6d85c78995-lvvwb                    1/1     Running   0          45s
csi-provisioner-6d85c78995-nkm7g                    1/1     Running   0          45s
csi-resizer-677c8b7c75-h9kc7                        1/1     Running   0          45s
csi-resizer-677c8b7c75-qbvxl                        1/1     Running   0          45s
csi-resizer-677c8b7c75-wppts                        1/1     Running   0          45s
csi-snapshotter-868c6c774d-4z57t                    1/1     Running   0          45s
csi-snapshotter-868c6c774d-9t2cd                    1/1     Running   0          45s
csi-snapshotter-868c6c774d-ps5lp                    1/1     Running   0          45s
engine-image-ei-b0369a5d-4jqzf                      1/1     Running   0          47s
instance-manager-84b0d2d8f37aaeabfa5785c886a57710   1/1     Running   0          47s
longhorn-csi-plugin-bsv4m                           3/3     Running   0          45s
longhorn-driver-deployer-6dff48688b-4497d           1/1     Running   0          62s
longhorn-manager-ft96r                              1/1     Running   0          62s
longhorn-ui-966cf4bb4-5shp9                         1/1     Running   0          62s
longhorn-ui-966cf4bb4-lx2th                         1/1     Running   0          62s

  8. By default, the longhorn storage class is set to 3 replicas for redundancy. This setting is stored at the longhorn-storageclass config map. If your k8s cluster has less than three nodes, set it to 1 replica:

    kubectl get cm longhorn-storageclass -n longhorn-system -o yaml | sed 's/numberOfReplicas: "3"/numberOfReplicas: "1"/' | kubectl apply -f -
configmap/longhorn-storageclass configured

  9. Setup a recurring job to trim the filesystem periodically to reclaim disk space, create a trim-filesystem.recurring-job.yaml file:

    apiVersion: longhorn.io/v1beta2
kind: RecurringJob
metadata:
  name: trim-filesystem
  namespace: longhorn-system
spec:
  concurrency: 1
  cron: 0 0 * * *
  name: trim-filesystem
  task: filesystem-trim
  groups:
    - default

  10. Create another recurring job to periodically purge removable snapshots snapshot-cleanup.recurring-job.yaml:

    apiVersion: longhorn.io/v1beta2
kind: RecurringJob
metadata:
  name: snapshot-cleanup
  namespace: longhorn-system
spec:
  concurrency: 1
  cron: 12 0 * * *
  groups:
    - default
  name: snapshot-cleanup
  task: snapshot-cleanup

  11. Deploy all tow recurring jobs:

    kubectl apply -f trim-filesystem.recurring-job.yaml 
recurringjob.longhorn.io/trim-filesystem created

kubectl apply -f snapshot-cleanup.recurring-job.yaml 
recurringjob.longhorn.io/snapshot-cleanup created

  12. By default, Longhorn does not delete orphan resources automatically, but it does provide the setting for us to do so, set orphan-auto-deletion to true:

    kubectl -n longhorn-system get setting orphan-auto-deletion -o yaml | sed 's/value: "false"/value: "true"/' | kubectl apply -f -
setting.longhorn.io/orphan-auto-deletion configured

kubectl -n longhorn-system get setting orphan-auto-deletion
NAME                   VALUE   AGE
orphan-auto-deletion   true    53m

Conclusion

Now that you have the Longhorn system installed in your Kubernetes cluster with a fine-tuned configuration. Consider following the Best Practices to improve your system further.

References

Comments