Kustomize

Kustomize

Kustomize traverses a Kubernetes manifest to add, remove or update configuration options without forking. It is available both as a standalone binary and as a native feature of kubectl.

https://kustomize.io/

https://kustomize.io/

Install

https://kubectl.docs.kubernetes.io/installation/kustomize/

Option 1 – homebrew

brew install kustomize

Option 2

https://github.com/kubernetes-sigs/kustomize/releases

wget https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv4.5.5/kustomize_v4.5.5_linux_amd64.tar.gz
tar zxf kustomize_v4.5.5_linux_amd64.tar.gz
rm kustomize_v4.5.5_linux_amd64.tar.gz
sudo mv kustomize /usr/local/bin

Alias

vi ~/.zshrc alias
ku="/usr/local/bin/kustomize"
source ~/.zshrc

Basic Demo

Create kustomize-demo directory.

mkdir kustomize-demo && cd kustomize-demo

Create deployment.

cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx EOF

Create a new kustomization detecting and adding all resources within the current directory.

kustomize create --autodetect
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml

Build.

Nothing has changed.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx

Add namespace.

kustomize edit set namespace itsmetommy
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
namespace: itsmetommy

Build.

Namespace is added.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
name: nginx
namespace: itsmetommy
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx

Add label.

kustomize edit set label env:dev
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
namespace: itsmetommy
commonLabels:
env: dev

Build.

Label added.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: dev
name: nginx
namespace: itsmetommy
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: dev
template:
metadata:
labels:
app: nginx
env: dev
spec:
containers:
- image: nginx
name: nginx

Add image.

kustomize edit set image nginx=nginx:1.21.6
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
namespace: itsmetommy
commonLabels:
env: dev
images:
- name: nginx
newName: nginx
newTag: 1.21.6

Build.

Image added.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: dev
name: nginx
namespace: itsmetommy
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: dev
template:
metadata:
labels:
app: nginx
env: dev
spec:
containers:
- image: nginx:1.21.6
name: nginx

Create deployment.

# Option 1 - use kustomize
kustomize build | kubectl apply -f -

# Option 2 - use kubectl
kubectl apply -k .

Clean up.

# Option 1 - use kustomize
kustomize build | kubectl delete -f -

# Option 2 - use kubectl
kubectl delete -k .

Standard Structure

Feel free to clone the git repo.

git clone https://github.com/itsmetommy/kustomize.git

Create a kustomize structure with best practices in mind using a base and overlays directory.

Create directories.

{
  mkdir -p kustomize-demo-best-practices/base
  mkdir kustomize-demo-best-practices/overlays
  mkdir kustomize-demo-best-practices/overlays/dev
  mkdir kustomize-demo-best-practices/overlays/prod
  mkdir kustomize-demo-best-practices/overlays/staging
  cd kustomize-demo-best-practices/base
}

Create base files.

cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx EOF

Create a new kustomization detecting and adding all resources within the current directory.

kustomize create --autodetect
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml

Build.

Nothing has changed.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx

cd into the overlays/dev directory.

cd ../overlays/dev

Create kustomization.yaml and include all files within the base directory.

Technically, you can create the kustomization.yaml file manually, but I am using kustomize commands so that you better understand your options.

kustomize create --resources ../../base

Add label env: dev.

kustomize edit set label env:dev

Add namesuffix.

kustomize edit set namesuffix -- -dev
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
commonLabels:
env: dev
nameSuffix: -dev

Build.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: dev
name: nginx-dev
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: dev
template:
metadata:
labels:
app: nginx
env: dev
spec:
containers:
- image: nginx

cd into the overlays/prod directory.

cd ../prod

Create kustomization.yaml and include all files within the base directory.

kustomize create --resources ../../base

Add label env: prod.

kustomize edit set label env:prod

Add namesuffix.

kustomize edit set namesuffix -- -prod
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
commonLabels:
env: prod
nameSuffix: -prod

Build.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: prod
name: nginx-prod
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: prod
template:
metadata:
labels:
app: nginx
env: prod
spec:
containers:
- image: nginx
name: nginx

cd into the overlays/staging directory.

cd ../staging

Create kustomization.yaml and include all files within the base directory.

kustomize create --resources ../../base

Add label env: staging.

kustomize edit set label env:staging

Add namesuffix.

kustomize edit set namesuffix -- -staging
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
commonLabels:
env: staging
nameSuffix: -staging

Build.

kustomize build
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: staging
name: nginx-staging
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: staging
template:
metadata:
labels:
app: nginx
env: staging
spec:
containers:
- image: nginx
name: nginx

cd back into the main directory.

cd ../..

Build each environment from the base directory.

kustomize build overlays/dev
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: dev
name: nginx-dev
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: dev
template:
metadata:
labels:
app: nginx
env: dev
spec:
containers:
- image: nginx
name: nginx
kustomize build overlays/prod
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: prod
name: nginx-prod
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: prod
template:
metadata:
labels:
app: nginx
env: prod
spec:
containers:
- image: nginx
name: nginx
kustomize build overlays/staging
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: staging
name: nginx-staging
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: staging
template:
metadata:
labels:
app: nginx
env: staging
spec:
containers:
- image: nginx
name: nginx

Add service.yaml to the base kustomization.yaml file.

cd base

Create a base service.

cat <<EOF > service.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  selector:
    app: nginx
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
EOF
kustomize edit add resource service.yaml
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml

Add service-patch.yaml to each environment.

cd ..

Create a dev service with a ClusterIP.

cat <<EOF > overlays/dev/service-patch.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  type: ClusterIP
EOF

Create a prod service with a LoadBalancer.

cat <<EOF > overlays/prod/service-patch.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  type: LoadBalancer
EOF

Create a staging service with a NodePort.

cat <<EOF > overlays/staging/service-patch.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    nodePort: 30080
    targetPort: 80
  type: NodePort
EOF

Add service-patch.yaml within patches to each kustomization.yaml file per environment.

cat <<EOF >> overlays/dev/kustomization.yaml
patches:
- service-patch.yaml
EOF
cat <<EOF >> overlays/prod/kustomization.yaml
patches:
- service-patch.yaml
EOF
cat <<EOF >> overlays/staging/kustomization.yaml
patches:
- service-patch.yaml
EOF

Build each environment.

kustomize build overlays/dev
apiVersion: v1
kind: Service
metadata:
labels:
env: dev
name: nginx-dev
spec:
selector:
env: dev
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: dev
name: nginx-dev
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: dev
template:
metadata:
labels:
app: nginx
env: dev
spec:
containers:
- image: nginx
name: nginx
kustomize build overlays/prod
apiVersion: v1
kind: Service
metadata:
labels:
env: prod
name: nginx-prod
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
env: prod
type: LoadBalancer
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: prod
name: nginx-prod
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: prod
template:
metadata:
labels:
app: nginx
env: prod
spec:
containers:
- image: nginx
name: nginx
kustomize build overlays/staging
apiVersion: v1
kind: Service
metadata:
labels:
env: staging
name: nginx-staging
spec:
ports:
- nodePort: 30080
port: 80
protocol: TCP
targetPort: 80
selector:
env: staging
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: staging
name: nginx-staging
spec:
replicas: 1
selector:
matchLabels:
app: nginx
env: staging
template:
metadata:
labels:
app: nginx
env: staging
spec:
containers:
- image: nginx
name: nginx

From here, you can apply whichever environment you want.

{
  kustomize build overlays/dev | kubectl apply -f -
  kustomize build overlays/prod | kubectl apply -f -
  kustomize build overlays/staging | kubectl apply -f -
}
kubectl get svc,deploy -l app=nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx-dev ClusterIP 10.96.196.237 <none> 80/TCP 33s
service/nginx-prod LoadBalancer 10.111.58.56 localhost 80:31018/TCP 61m
service/nginx-staging NodePort 10.96.11.203 <none> 80:30080/TCP 61m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-dev 1/1 1 1 61m
deployment.apps/nginx-prod 1/1 1 1 61m
deployment.apps/nginx-staging 1/1 1 1 61m

Directory tree.

tree
.
├── base
│   ├── deployment.yaml
│   ├── kustomization.yaml
│   └── service.yaml
└── overlays
    ├── dev
    │   ├── kustomization.yaml
    │   └── service-patch.yaml
    ├── prod
    │   ├── kustomization.yaml
    │   └── service-patch.yaml
    └── staging
        ├── kustomization.yaml
        └── service-patch.yaml

5 directories, 9 files

Clean up.

{
  kustomize build overlays/dev | kubectl delete -f -
  kustomize build overlays/prod | kubectl delete -f -
  kustomize build overlays/staging | kubectl delete -f -
}

There’s more to kustomize, but this should get you started.

e.g.

Create a kustomization.yaml file with the basic apiVersion and kind.

kustomize init
cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

Different ways to deploy.

# Option 1 - use kustomize
kustomize build | kubectl apply -f -

# Option 2 - use kubectl
kubectl apply -k .