Harry Tang

FluxCD with Multi-tenancy Setup

Flux multi-tenancy
Flux multi-tenancy

Introduction

Suppose you have different projects and/or teams and only one K8s cluster. In that case, you can follow this tutorial to create tenants for these projects and/or teams in an isolated environment using the Flux multi-tenancy feature.

Prerequisites

  • A bootstrapped FluxCD K8s cluster.
  • Flux CLI installed.
  • SealedSecret setup for K8s.
  • Basic knowledge of Kubernetes.

Step-by-step Guide

These steps describe how to create two environments, staging and production, for the project called Harry Tang; each environment is a tenant and also is a K8s namespace:

  1. Create a folder for the tenant inside the Flux's git repository:

    export NAMESPACE='YOUR_NAMESPACE' # e.g. harrytang-staging
mkdir -p $NAMESPACE

  2. Use Flux CLI to create the tenant:

    flux create tenant $NAMESPACE --with-namespace=$NAMESPACE --export > ./$NAMESPACE/tenant.yaml

  3. Create a git source for the tenant:

    export GIT_SOURCE='YOUR_GIT_SOURCE' # e.g. harrytang-infra
export GIT_URL='YOUR_GIT_URL' # e.g. ssh://[email protected]/harrytang/infra
flux create source git $GIT_SOURCE \
 --namespace=$NAMESPACE \
 --secret-ref=$GIT_SOURCE \
 --url=$GIT_URL \
 --branch=main \
 --export > ./$NAMESPACE/sync.yaml

  4. Create kustomization:

    export KPATH='YOUR_PATH' # e.g. ./apps/production
flux create kustomization $GIT_SOURCE \
  --namespace=$NAMESPACE \
  --service-account=$NAMESPACE \
  --source=GitRepository/$GIT_SOURCE \
  --path=$KPATH \
  --export >> ./$NAMESPACE/sync.yaml

  5. Create a git secret for the tenant:

    flux create secret git $GIT_SOURCE \
  --namespace=$NAMESPACE \
  --url=$GIT_URL \
  --ssh-key-algorithm=ecdsa \
  --ssh-ecdsa-curve=p521 \
  --export | kubeseal --format yaml > ./$NAMESPACE/sealed-secret.yaml

  6. Create a service account token service-account-token.yaml:

    apiVersion: v1
kind: Secret
metadata:
  name: YOUR_NAMESPACE # replace with $NAMESPACE
  namespace: YOUR_NAMESPACE # replace with $NAMESPACE
  annotations:
    kubernetes.io/service-account.name: YOUR_NAMESPACE # replace with $NAMESPACE
type: kubernetes.io/service-account-token

  7. Repeat these steps for the production environment:

    export NAMESPACE='YOUR_NAMESPACE' # e.g. harrytang-prod
mkdir -p $NAMESPACE

    Repeat steps 2-7.

Finally, commit and push your changes, and magic will happen.

Conclusion

FluxCD multi-tenancy allows you to manage different projects/teams with different isolated environments easily. It is also considered a best practice for managing a K8s cluster.

References

Comments