Hero Image
- Philipp Ludewig

How to exchange storage classes in kubernetes

Aloha people,

this tutorial will explain how to migrate data from one StorageClass to another.

Here is the scenario: You have a certain setup with some prometheus pods. These pods have a PersitanceVolumeClaim (pvc) of 10Gi and a StorageClass (sc) called "gp2" with an PersistanceVolume (pv) attached to it. Now the thing is the sc was manually configured to use 'retain' as the reclaimPolicy. This will be resettet back to 'delete' with deploying the same cluster again. We now want to move the persistence volumes to a new defined sc with the reclaimPolicy set to 'retain'. The important fact is that we don't want to lose the data of these volumes.

So here without further ado… here… we… go.

Step 1: List your pods and find out which pods to scale down

kubectl get pods -n namespace

Step 2: List your deployments, replicasets or statefulsets

kubectl get statefulset -n namespace

Step 3: Scale down the pods which are writing to the volumes

kubectl scale statefulset --replicas 0 -n namespace name_of_statfulset

Step 4: List your storage classes

kubectl get sc -n namespace

Step 5: Create a new storage class

kubectl apply -f storage.yml

Here is a example configuration for a storage class:

[here is your current storage class...]


kind: StorageClass
apiVersion: storage.k8s.io/v1
  name: [NAME_OF_SC]
provisioner: kubernetes.io/aws-ebs
  type: gp2
reclaimPolicy: Retain
  - debug

Step 6: tell the persistence volume to use the new storage class

  1. go into the pv configuration on kubernetes
    kubectl edit pv -n namespace pvc-uuid_of_pv
  2. change field 'storageClassName' to the name of the new sc

Step 7: create local file with configuration of pvc and delete all unneccessary lines

kubectl edit pvc -n namespace name_of_pvc

Here is a sample configuration:

apiVersion: v1
kind: PersistentVolumeClaim
    pv.kubernetes.io/bind-completed: "yes"
    pv.kubernetes.io/bound-by-controller: "yes"
    volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs
  - kubernetes.io/pvc-protection
    app: prometheus
    prometheus: k8s
  name: [NAME_OF_PVC]
  namespace: monitoring
  - ReadWriteOnce
  dataSource: null
      storage: 10Gi
  storageClassName: gp2-retain
  volumeName: [NAME_OF_PV]
  - ReadWriteOnce
    storage: 10Gi
  phase: Bound

Step 8: delete the pvc which you want to switch but remember the corresponding name_of_pv

kubectl delete pvc -n namespace name_of_pvc

Step 9: create new pvc which is using the new sc

kubectl apply -n namespace -f the_config_file_of_the_old_pvc

If you now query the pvc the newly created one should display as being lost. This is fine because now we need to connect the pv which is already connected to the sc with it's corresponding pvc.

Step 10: Connect pv with corresponding PersitanceVolumeClaim

  1. Open the config of the newly created pvc
    kubectl edit pvc -n namespace name_of_pvc
  2. copy the uid of the pvc
    app: ...
    prometheus: ...
    name: name_of_pvc
    resourceVersion: ...
    selflink: ...
    uid: 871e49b9-3dbe-48d3-b1e8-c2439ec1fb78 
  3. Edit the pv and change the uid of the corresponding pvc in the config

    kubectl get pv -n namespace
    kubectl edit pv -n namespace name_of_pv
    Search and change this:
    apiVersion: ...
    kind: PersitanceVolumeClaim
    name: name_of_pvc
    namespace: namespace
    resourceVersion: ...
    uid: [uuid_of_pvc]

Step 11: Scale up your pods again

  kubectl scale statefulset --replicas 2 -n namespace name_of_statfulset

AAAND now you are done :)