Kubernetes: Migrate Local Storage to Google Cloud Storage Bucket

I ran into an issue where having a local disk wasn’t the best solution and decided it was time to migrate to a Google Cloud Storage Bucket.

This particular situation has to do with Artifactory where I was using a PersistentVolume (gcePersistentDisk) and now wanted to use a storage bucket (the right way).

I had about 1.9TB of data that needed to migrate to a storage bucket.

kubectl exec -it artifactory-0 -- du -d 1 -h /var/opt/jfrog/artifactory/data/
1.9T /var/opt/jfrog/artifactory/data/

Create a bucket

GCS_BUCKET_NAME=[YOUR_BUCKET_NAME]
gsutil mb gs://${GCS_BUCKET_NAME}

Create Google Service Account

Create a Google Service Account and generate a JSON key with Storage Admin permissions. This will be used to access the Storage Bucket from within the pod.

{
   SERVICE_ACCOUNT_NAME=artifactory-migration-sa
   SERVICE_ACCOUNT_DEST=artifactory-migration-sa.json
   SERVICE_ACCOUNT_DISPLAY_NAME="Artifactory Migration Storage Account"
   PROJECT="$(gcloud info --format='value(config.project)')"
 # Create Service Account
   gcloud iam service-accounts create \
     $SERVICE_ACCOUNT_NAME \
     --project $PROJECT \
     --display-name $SERVICE_ACCOUNT_DISPLAY_NAME
 sleep 10
 # List Service Account Email
   SA_EMAIL=$(gcloud iam service-accounts list \
     --project=$PROJECT \
     --filter="email ~ $SERVICE_ACCOUNT_NAME" \
     --format='value(email)')
 # Associate Role
   gcloud projects add-iam-policy-binding \
     $PROJECT \
     --role roles/storage.admin \
     --member serviceAccount:$SA_EMAIL
 # Download Service Account key
   gcloud iam service-accounts keys create \
     $SERVICE_ACCOUNT_DEST \
     --project $PROJECT \
     --iam-account $SA_EMAIL
 }

Create a secret with the Service Account credentials

Create the Secret containing the Service Account which enables authentication to Google Cloud Storage.

kubectl create secret generic artifactory-migration-key \
   --from-file=key.json=./artifactory-migration-sa.json

Update statefulset

Mount the key within the container google-cloud-sdk. I used the google-cloud-sdk image because it comes with gsutil.

kubectl edit statefuleset artifactory
 …
     spec:
       containers:
         - name: google-cloud-sdk
           image: gcr.io/google.com/cloudsdktool/cloud-sdk
           command: ["/bin/bash","-c","while true; do sleep 1000; done"]
           env:
             - name: GOOGLE_APPLICATION_CREDENTIALS
               value: /var/secret/google/key.json
           volumeMounts:
             - name: artifactory-migration-key
               mountPath: /var/secret/google
       volumes:
         - name: artifactory-migration-key
           secret:
             secretName: "artifactory-migration-key"

Copy data to bucket

kubectl exec -it artifactory-0 -- sh

Copy the data.

gsutil -m rsync -r $JFROG_HOME/artifactory/data/filestore gs://[YOUR_BUCKET_NAME]/artifactory/filestore

Example

gsutil -m rsync -r /var/opt/jfrog/artifactory/data/filestore gs://[YOUR_BUCKET_NAME]/artifactory/filestore

You should now see that your data is being migrated to the bucket.

Verify

gsutil ls gs://[YOUR_BUCKET_NAME]