It's Me, Tommy

Kubernetes: Cluster Backup with Heptio Velero on GCP

Heptio Ark is now Velero! As I’ve already written HERE, I thought I’d give an update to the slightly different installation of Heptio Velero.

This is going to be more of a cliff notes blog, as the previous blog has a lot of information I’d rather not repeat in this one. So here we go.

Install Client

Option 1

I got an error, but I assume it will work in the future.

brew install velero

Error

brew install velero
 Error: No available formula with the name "velero"
 ==> Searching for a previously deleted formula (in the last month)…
 Warning: homebrew/core is shallow clone. To get complete history run:
   git -C "$(brew --repo homebrew/core)" fetch --unshallow
 Error: No previously deleted formula found.
 ==> Searching for similarly named formulae…
 Error: No similarly named formulae found.
 ==> Searching taps…
 ==> Searching taps on GitHub…
 Error: No formulae found in taps.

Option 2

View latest release https://github.com/heptio/velero/releases.

The following commands downloads the Velero binary and copies it to /usr/local/bin.

{
  VERSION=v0.11.0
  OS=$(uname | tr '[:upper:]' '[:lower:]')
  ARCH=amd64
  curl -L https://github.com/heptio/velero/releases/download/$VERSION/velero-$VERSION-$OS-$ARCH.tar.gz \
    -o velero-$VERSION-$OS-$ARCH.tar.gz
  tar -zxvf velero-$VERSION-$OS-$ARCH.tar.gz velero
  chmod +x ./velero
  sudo mv ./velero /usr/local/bin/velero
  rm -rf velero-$VERSION-$OS-$ARCH.tar.gz
}

Install Server

Download the config directory, which holds all the files required to install velero onto your cluster.

The following commands creates a directory called heptio-velero and downloads the config directory into it (keeps things more organized).

{
  VERSION=v0.11.0
  OS=$(uname | tr '[:upper:]' '[:lower:]')
  ARCH=amd64
  mkdir heptio-velero && cd heptio-velero
  curl -L https://github.com/heptio/velero/releases/download/$VERSION/velero-$VERSION-$OS-$ARCH.tar.gz \
    -o velero-$VERSION-$OS-$ARCH.tar.gz
  tar -zxvf velero-$VERSION-$OS-$ARCH.tar.gz config
  rm -rf velero-$VERSION-$OS-$ARCH.tar.gz
}

Run on GCP

Create GCS bucket

BUCKET=[YOUR_BUCKET]

Create service account

Store the project value from the results in the environment variable $PROJECT_ID.

PROJECT_ID=$(gcloud config get-value project)

Create service account.

gcloud iam service-accounts create velero \
  --display-name "Velero service account"

Set the $SERVICE_ACCOUNT_EMAIL variable to match its email value.

SERVICE_ACCOUNT_EMAIL=velero@ace-element-175818.iam.gserviceaccount.com

Attach policies to give Velero the necessary permissions to function:

A

ROLE_PERMISSIONS=(
 compute.disks.get
 compute.disks.create
 compute.disks.createSnapshot
 compute.snapshots.get
 compute.snapshots.create
 compute.snapshots.useReadOnly
 compute.snapshots.delete
 compute.projects.get
)

B

gcloud iam roles create velero.server \
  --project $PROJECT_ID \
  --title "Velero Server" \
  --permissions "$(IFS=","; echo "${ROLE_PERMISSIONS[*]}")"

C

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT_EMAIL \
  --role projects/$PROJECT_ID/roles/velero.server

D

gsutil iam ch serviceAccount:$SERVICE_ACCOUNT_EMAIL:objectAdmin gs://${BUCKET}

Create a service account key, specifying an output file (credentials-velero) in your local directory:

Note: This will download the file credentials-velero to your localhost.

gcloud iam service-accounts keys create credentials-velero \
  --iam-account $SERVICE_ACCOUNT_EMAIL

Credentials and configuration

Prereqs

kubectl apply -f config/common/00-prereqs.yaml

Create a Secret

kubectl create secret generic cloud-credentials \
  --namespace velero \
  --from-file cloud=credentials-velero

Start the server

{
  kubectl apply -f config/gcp/05-backupstoragelocation.yaml
  kubectl apply -f config/gcp/06-volumesnapshotlocation.yaml
  kubectl apply -f config/gcp/10-deployment.yaml
}

Testing

I’m using namespace itsmetommy-test to test backup situations.

# create namespace
kubectl create ns itsmetommy-test

# create a deployment
kubectl create deployment nginx --image=nginx -n itsmetommy-test

# confirm deployment
kubectl get deploy -n itsmetommy-test

velero backup create itsmetommy-test-ns --include-namespaces itsmetommy-test --snapshot-volumes

# confirm backup
velero get backups

# simulate disaster
kubectl delete ns itsmetommy-test

# confirm namespace is gone
k get ns itsmetommy-test

# restore
velero restore create --from-backup itsmetommy-test-ns

# confirm restore
velero get restores

# confirm deployment restored
kubectl get deploy -n itsmetommy-test

# clean up
{
  kubectl delete ns itsmetommy-test-ns
  sleep 60
  velero backup delete itsmetommy-test-ns
}

Tommy Elmesewdy

,
, , , ,