Create IAM Service Account
- https://www.spinnaker.io/setup/quickstart/halyard-gke-deploy-rbac/#part-2-add-service-account-to-gcp
Spinnaker will use this IAM Service Account to access Google Cloud Storage (GCS) (storage.admin).
export SERVICE_ACCOUNT_NAME=spinnaker-itsmetommy-sa
export SERVICE_ACCOUNT_FILE=spinnaker-itsmetommy-sa.json export SERVICE_ACCOUNT_DISPLAY_NAME="Spinnaker Account"
export PROJECT=$(gcloud info --format='value(config.project)')
gcloud --project ${PROJECT} iam service-accounts create \
${SERVICE_ACCOUNT_NAME} \
--display-name ${SERVICE_ACCOUNT_DISPLAY_NAME}
sleep 10 SA_EMAIL=$(gcloud iam service-accounts list \ --project=${PROJECT} \
--filter="email ~ ${SERVICE_ACCOUNT_NAME}" \
--format='value(email)')
gcloud --project ${PROJECT} projects add-iam-policy-binding ${PROJECT} \
--role roles/storage.admin --member serviceAccount:${SA_EMAIL}
gcloud --project ${PROJECT} iam service-accounts keys create ${SERVICE_ACCOUNT_FILE} \
--iam-account ${SA_EMAIL}
Create namespace
kubectl create ns spinnaker
Create k8s Service Account
This Service Account gives the Spinnaker deployment admin permissions to the cluster.
cat <<EOF | kubectl create -f - apiVersion: v1
kind: ServiceAccount
metadata:
name: spinnaker-itsmetommy-sa
namespace: spinnaker
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: spinnaker-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: spinnaker-itsmetommy-sa
namespace: spinnaker EOF
Create k8s kubeconfig
Create a kubeconfig file based on the above k8s Service Account (admin access to the Spinnaker namespace). We will use it when we configure our Halyard kubernetes provider below.
SERVICE_ACCOUNT_NAME=spinnaker-itsmetommy-sa CONTEXT=$(kubectl config current-context) SA_NAMESPACE=spinnaker GCP_PROJECT=$(gcloud info --format='value(config.project)') KUBECONFIG_FILE=spinnaker-itsmetommy-kubeconfig NEW_CONTEXT=${SA_NAMESPACE}-sa SECRET_NAME=$(kubectl get serviceaccount ${SERVICE_ACCOUNT_NAME} \ --context ${CONTEXT} \ --namespace ${SA_NAMESPACE} \ -o jsonpath='{.secrets[0].name}') TOKEN_DATA=$(kubectl get secret ${SECRET_NAME} \ --context ${CONTEXT} \ --namespace ${SA_NAMESPACE} \ -o jsonpath='{.data.token}') case "$(uname -s)" in Darwin*) TOKEN=$(echo ${TOKEN_DATA} | base64 -D);; Linux*) TOKEN=$(echo ${TOKEN_DATA} | base64 -d);; *) TOKEN=$(echo ${TOKEN_DATA} | base64 -d);; esac kubectl config view --raw > ${KUBECONFIG_FILE}.full.tmp # Switch working context to correct context kubectl --kubeconfig ${KUBECONFIG_FILE}.full.tmp config use-context ${CONTEXT} # Minify kubectl --kubeconfig ${KUBECONFIG_FILE}.full.tmp \ config view --flatten --minify > ${KUBECONFIG_FILE}.tmp # Rename context kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \ rename-context ${CONTEXT} ${NEW_CONTEXT} # Create token user kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \ set-credentials ${CONTEXT}-${SA_NAMESPACE}-token-user \ --token ${TOKEN} # Set context to use token user kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \ set-context ${NEW_CONTEXT} --user ${CONTEXT}-${SA_NAMESPACE}-token-user # Set context to correct namespace kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \ set-context ${NEW_CONTEXT} --namespace ${SA_NAMESPACE} # Flatten/minify kubeconfig kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \ view --flatten --minify > ${KUBECONFIG_FILE} # Remove tmp rm ${KUBECONFIG_FILE}.full.tmp rm ${KUBECONFIG_FILE}.tmp
Test the kubeconfig file.
kubectl --kubeconfig=spinnaker-itsmetommy-kubeconfig get pods
Create Halyard GCE Persistent Disk
gcloud compute disks create --size=200GB --zone=us-west1-a halyard-itsmetommy
Create k8s Halyard Deployment
Notice the Service Account spinnaker-itsmetommy-sa. It gives the Halyard deployment admin permissions to the cluster.
Get the latest stable version from https://gcr.io/spinnaker-marketplace/halyard.
kubectl apply -f - <<EOF apiVersion: apps/v1
kind: Deployment
metadata:
name: halyard
namespace: spinnaker
labels:
app: halyard
spec:
replicas: 1 strategy:
type: Recreate
selector:
matchLabels:
app: halyard
template:
metadata:
labels:
app: halyard
spec: nodeSelector:
failure-domain.beta.kubernetes.io/zone: us-west1-a
serviceAccountName: spinnaker-itsmetommy-sa
containers:
- name: halyard
image: gcr.io/spinnaker-marketplace/halyard:1.42.0
volumeMounts:
- name: halyard-itsmetommy
mountPath: "/home/spinnaker" securityContext:
fsGroup: 1000 runAsUser: 1000
volumes:
- name: halyard-itsmetommy
gcePersistentDisk:
pdName: halyard-itsmetommy
fsType: ext4 EOF
Create GCS bucket for Spinnaker
We want persistent data generated by Spinnaker (e.g. pipeline definitions), so we will create a GCS bucket. We will configure the storage bucket in the next step.
BUCKET=[BUCKET_NAME] gsutil mb gs://$BUCKET
Connect to the Halyard Pod
kubectl exec -it `kubectl get pods -n spinnaker -l app=halyard -o jsonpath="{.items[0].metadata.name}"` -n spinnaker -- sh
Configure Persistent Storage for Spinnaker
Because we want persistent data generated by Spinnaker (e.g. pipeline definitions), we will now configure Spinnaker to use the bucket we created in the previous step.
I decided to create a folder structure [PROVIDER]/[PROJECT] to keep things organized within the .secret directory..
# Make directory GCP_PROJECT=[PROJECT_NAME] mkdir -p /home/spinnaker/.secret/gcp/$GCP_PROJECT
Copy spinnaker-itsmetommy-sa.json to the Halyard pod.
GCP_PROJECT=[PROJECT_NAME] HALYARD_POD=$(kubectl get pods -n spinnaker -l app=halyard -o jsonpath="{.items[0].metadata.name}") kubectl cp spinnaker-itsmetommy-sa.json $HALYARD_POD:/home/spinnaker/.secret/gcp/$GCP_PROJECT -n spinnaker # Verify kubectl exec $HALYARD_POD -n spinnaker -- ls /home/spinnaker/.secret/gcp/$GCP_PROJECT
Configure the storage bucket.
GCP_PROJECT=[PROJECT_NAME] BUCKET_LOCATION=us BUCKET=spinnaker-itsmetommy SERVICE_ACCOUNT_FILE=/home/spinnaker/.secret/gcp/$GCP_PROJECT/spinnaker-itsmetommy-sa.json hal config storage gcs edit \ --project $GCP_PROJECT \ --bucket-location $BUCKET_LOCATION \ --bucket $BUCKET \ --json-path $SERVICE_ACCOUNT_FILE \ \ && hal config storage edit --type gcs
Configure Kubernetes Provider
Copy spinnaker-itsmetommy-kubeconfig to the Halyard pod.
GCP_PROJECT=[PROJECT_NAME] HALYARD_POD=$(kubectl get pods -n spinnaker -l app=halyard -o jsonpath="{.items[0].metadata.name}") kubectl cp spinnaker-itsmetommy-kubeconfig $HALYARD_POD:/home/spinnaker/.secret/gcp/$GCP_PROJECT -n spinnaker
Configure kubeconfig and Cloud Provider.
GCP_PROJECT=[PROJECT_NAME] SA_NAMESPACE=spinnaker ACCOUNT_NAME=k8s-itsmetommy KUBECONFIG_FILE=/home/spinnaker/.secret/gcp/$GCP_PROJECT/spinnaker-itsmetommy-kubeconfig # Enable the Kubernetes cloud provider hal config provider kubernetes enable # Add the account hal config provider kubernetes account add $ACCOUNT_NAME \ --provider-version v2 \ --kubeconfig-file $KUBECONFIG_FILE \ --only-spinnaker-managed true \ --namespaces $SA_NAMESPACE # Verify hal config provider kubernetes account list
Configure Artifacts
Within Spinnaker, ‘artifacts’ are consumable references to items that live outside of Spinnaker (for example, a file in a git repository or a file in an S3 bucket are two examples of artifacts). This feature must be explicitly turned on.
hal config features edit --artifacts true
Configure Distributed
Halyard deploys each of Spinnaker’s microservices separately. This is highly recommended for use in production.
Distributed installations are for development orgs with large resource footprints, and for those who can’t afford downtime during Spinnaker updates.
# Distributed installations are for development orgs with large resource footprints, # and for those who can’t afford downtime during Spinnaker updates. ACCOUNT_NAME=k8s-itsmetommy hal config deploy edit --type distributed --account-name $ACCOUNT_NAME
Deploy Spinnaker
Choose Spinnaker version
# latest hal config version edit --version $(hal version latest -q) # Or you can specify the version hal version list export VERSION=1.23.0 hal config version edit --version $VERSION
Specify namespace (optional)
The default namespace is spinnaker.
hal config deploy edit --location [NAMESPACE]
Install Spinnaker
hal deploy apply
Watch
The complete spinnaker installation will take ~10 minutes.
watch kubectl get pods -n spinnaker
Connect to Spinnaker
- Deck: Management UI for Spinnaker
- Gate: Spinnaker API Gateway
NAMESPACE=spinnaker
DECK_POD=$(kubectl -n ${NAMESPACE} get pod -l cluster=spin-deck -ojsonpath='{.items[0].metadata.name}')
GATE_POD=$(kubectl -n ${NAMESPACE} get pod -l cluster=spin-gate -ojsonpath='{.items[0].metadata.name}')
kubectl -n ${NAMESPACE} port-forward ${DECK_POD} 9000 &
kubectl -n ${NAMESPACE} port-forward ${GATE_POD} 8084 &
View jobs.
jobs
[1] - running kubectl -n ${NAMESPACE} port-forward ${DECK_POD} 9000
[2] + running kubectl -n ${NAMESPACE} port-forward ${GATE_POD} 8084
Open browser.
open http://localhost:9000

Clean up
# Kill jobs kill %1 %2 # Delete Spinnaker deployment kubectl exec -it `kubectl get pods -n spinnaker -l app=halyard -o jsonpath="{.items[0].metadata.name}"` -n spinnaker -- sh hal deploy clean # Delete IAM Service Account export SERVICE_ACCOUNT_NAME=spinnaker-itsmetommy-sa export PROJECT=$(gcloud info --format='value(config.project)') gcloud --project ${PROJECT} iam service-accounts delette ${SERVICE_ACCOUNT_NAME} rm spinnaker-itsmetommy-sa.json # Delete k8s kubeconfig file rm spinnaker-itsmetommy-kubeconfig # Delete Halyard deployment kubectl delete deployment halyard -n spinnaker # Delete GCE Persistent Disk gcloud compute disks delete --zone=us-west1-a halyard-itsmetommy # Delete bucket gsutil rm -r gs://[BUCKET_NAME]