FluxCD with Multi-tenancy Setup
Introduction
Suppose you have different projects and/or teams and only one K8s cluster. In that case, you can follow this tutorial to create tenants for these projects and/or teams in an isolated environment using the Flux multi-tenancy feature.
Prerequisites
- A bootstrapped FluxCD K8s cluster.
- Flux CLI installed.
- SealedSecret setup for K8s.
- Basic knowledge of Kubernetes.
Step-by-step Guide
These steps describe how to create two environments, staging and production, for the project called Harry Tang
; each environment is a tenant and also is a K8s namespace:
-
Create a folder for the tenant inside the Flux's git repository:
export NAMESPACE='YOUR_NAMESPACE' # e.g. harrytang-staging mkdir -p $NAMESPACE
-
Use Flux CLI to create the tenant:
flux create tenant $NAMESPACE --with-namespace=$NAMESPACE --export > ./$NAMESPACE/tenant.yaml
-
Create a git source for the tenant:
export GIT_SOURCE='YOUR_GIT_SOURCE' # e.g. harrytang-infra export GIT_URL='YOUR_GIT_URL' # e.g. ssh://[email protected]/harrytang/infra flux create source git $GIT_SOURCE \ --namespace=$NAMESPACE \ --secret-ref=$GIT_SOURCE \ --url=$GIT_URL \ --branch=main \ --export > ./$NAMESPACE/sync.yaml
-
Create kustomization:
export KPATH='YOUR_PATH' # e.g. ./apps/production flux create kustomization $GIT_SOURCE \ --namespace=$NAMESPACE \ --service-account=$NAMESPACE \ --source=GitRepository/$GIT_SOURCE \ --path=$KPATH \ --export >> ./$NAMESPACE/sync.yaml
-
Create a git secret for the tenant:
flux create secret git $GIT_SOURCE \ --namespace=$NAMESPACE \ --url=$GIT_URL \ --ssh-key-algorithm=ecdsa \ --ssh-ecdsa-curve=p521 \ --export | kubeseal --format yaml > ./$NAMESPACE/sealed-secret.yaml
-
Create a service account token
service-account-token.yaml
:apiVersion: v1 kind: Secret metadata: name: YOUR_NAMESPACE # replace with $NAMESPACE namespace: YOUR_NAMESPACE # replace with $NAMESPACE annotations: kubernetes.io/service-account.name: YOUR_NAMESPACE # replace with $NAMESPACE type: kubernetes.io/service-account-token
-
Repeat these steps for the
production
environment:export NAMESPACE='YOUR_NAMESPACE' # e.g. harrytang-prod mkdir -p $NAMESPACE
Repeat steps 2-7.
Finally, commit and push your changes, and magic will happen.
Conclusion
FluxCD multi-tenancy allows you to manage different projects/teams with different isolated environments easily. It is also considered a best practice for managing a K8s cluster.