I ran into an issue where having a local disk wasn’t the best solution and decided it was time to migrate to a Google Cloud Storage Bucket.
This particular situation has to do with Artifactory where I was using a PersistentVolume (gcePersistentDisk) and now wanted to use a storage bucket (the right way).
I had about 1.9TB of data that needed to migrate to a storage bucket.
kubectl exec -it artifactory-0 -- du -d 1 -h /var/opt/jfrog/artifactory/data/ 1.9T /var/opt/jfrog/artifactory/data/
Create a bucket
GCS_BUCKET_NAME=[YOUR_BUCKET_NAME] gsutil mb gs://${GCS_BUCKET_NAME}
Create Google Service Account
Create a Google Service Account and generate a JSON key with Storage Admin permissions. This will be used to access the Storage Bucket from within the pod.
{ SERVICE_ACCOUNT_NAME=artifactory-migration-sa SERVICE_ACCOUNT_DEST=artifactory-migration-sa.json SERVICE_ACCOUNT_DISPLAY_NAME="Artifactory Migration Storage Account" PROJECT="$(gcloud info --format='value(config.project)')" # Create Service Account gcloud iam service-accounts create \ $SERVICE_ACCOUNT_NAME \ --project $PROJECT \ --display-name $SERVICE_ACCOUNT_DISPLAY_NAME sleep 10 # List Service Account Email SA_EMAIL=$(gcloud iam service-accounts list \ --project=$PROJECT \ --filter="email ~ $SERVICE_ACCOUNT_NAME" \ --format='value(email)') # Associate Role gcloud projects add-iam-policy-binding \ $PROJECT \ --role roles/storage.admin \ --member serviceAccount:$SA_EMAIL # Download Service Account key gcloud iam service-accounts keys create \ $SERVICE_ACCOUNT_DEST \ --project $PROJECT \ --iam-account $SA_EMAIL }
Create a secret with the Service Account credentials
Create the Secret containing the Service Account which enables authentication to Google Cloud Storage.
kubectl create secret generic artifactory-migration-key \ --from-file=key.json=./artifactory-migration-sa.json
Update statefulset
Mount the key within the container google-cloud-sdk. I used the google-cloud-sdk image because it comes with gsutil.
kubectl edit statefuleset artifactory … spec: containers: - name: google-cloud-sdk image: gcr.io/google.com/cloudsdktool/cloud-sdk command: ["/bin/bash","-c","while true; do sleep 1000; done"] env: - name: GOOGLE_APPLICATION_CREDENTIALS value: /var/secret/google/key.json volumeMounts: - name: artifactory-migration-key mountPath: /var/secret/google volumes: - name: artifactory-migration-key secret: secretName: "artifactory-migration-key"
Copy data to bucket
Login to the pod.
kubectl exec -it artifactory-0 -- sh
Copy the data.
gsutil rsync -r $JFROG_HOME/artifactory/data/filestore gs://[YOUR_BUCKET_NAME]
Example
gsutil rsync -r /var/opt/jfrog/artifactory/data/filestore gs://[YOUR_BUCKET_NAME]
You should now see that your data is being migrated to the bucket.

Verify
gsutil ls gs://[YOUR_BUCKET_NAME]