Volume Provisioning

The first thing to make use of storage is to provision the volumes. And one might need to provision more than one volumes to fit different use cases. In K8S, there two ways to get a volume provisioned: Static provisioning and Dynamic provisioning.

Static provisioning

• Provision the storage manually by following the instruction from the storage vendor.

If you are using VMware vSphere, you could create VMDK on your vSan datastore

* ssh to your esxi host
[root@esxi-dell-h:/vmfs/volumes/vsan:52fae366e94edb86-c6633d0af03e5aec] mkdir demo
[root@esxi-dell-h:/vmfs/volumes/vsan:52fae366e94edb86-c6633d0af03e5aec] cd demo
[root@esxi-dell-h:/vmfs/volumes/vsan:52fae366e94edb86-c6633d0af03e5aec/9cc4ef5c-370b-7eeb-876d-246e962c2408] vmkfstools -c 2G -d thin -W vsan demo.vmdk
Create: 100% done.
[root@esxi-dell-h:/vmfs/volumes/vsan:52fae366e94edb86-c6633d0af03e5aec/9cc4ef5c-370b-7eeb-876d-246e962c2408] ls
demo.vmdk

• Next step is to create the PV which consumes the storage you just created.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: demo-pv
spec:
  storageClassName: demo
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  vsphereVolume:
    volumePath: "[vsanDatastore] demo/demo.vmdk"
    fsType: ext4

storageClass: demois used to bind PVC to this PV. You could have no such StorageClass created before hand.

vsphereVolume is the in-tree volume plugin supported by K8S. More details could be found here

• Next step is to create PVC which could consume PV for a pod.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: demo-pvc
spec:
  storageClassName: demo
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

If the storageClassName does not match the one you used for PV, your PVC will not bind to the PV you just created.

• Use it in Pod or Deployment. The following YAML is just an example for Pod.

apiVersion: v1
kind: Pod
metadata:
  name: demo-pod
spec:
  containers:
  - name: busybox
    image: "k8s.gcr.io/busybox"
    volumeMounts:
    - name: demo-vol
      mountPath: "/demo"
    command: [ "sleep", "1000000" ]
  volumes:
    - name: demo-vol
      persistentVolumeClaim:
        claimName: demo-pvc

Follow the tutorial here if you would like to use minikube to do some experiments.

Above is the steps you would use to manually provision a storage/volume.

Dynamic provisioning

Why dynamic provision

• Before dynamic provisioning, cluster administrators had to manually make calls to their cloud or storage provider to provision new storage volumes, and then create PersistentVolume objects to represent them in Kubernetes. With dynamic provisioning, these two steps are automated, eliminating the need for cluster administrators to pre-provision storage. Instead, the storage resources can be dynamically provisioned using the provisioner specified by the StorageClass object.

• Better resource management ?

End to end workflow (Bottom up)

• Create StorageClass

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: demo-sc-vsan
provisioner: kubernetes.io/vsphere-volume
parameters:
  storagePolicyName: gold
  datastore: vsanDatastore

The provisioner specifies what volume plugin to be used for dynamic provisioning. The volume plugin is in charge of create and setup the VMDK in the vSphere example, and also create the PV automatically. Differnt vendor has different implementations.

• The provisioner has kubernetes.io as prefix is the in-tree volume plugin. Its implementation could be found at kubernetes/pkg/volume/vsphere_volume/vsphere_volume.go from https://github.com/kubernetes/kubernetes.git

• The provisioner used above is the in-tree provisioner, more about provisioner for the external storage provisioner could be found here.

• What is CSI could be found here.

• Create PVC

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: demo-sc-pvc
spec:
  storageClassName: demo-sc-vsan
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Once the PVC is created, it will find the storageclass for dynamic provisioning. The storageClassName must match the StorageClass created in step #1.

The design of volume provisioning could be found here.

• Pod or Deployment uses the PVC

apiVersion: v1
kind: Pod
metadata:
  name: demo-sc-pod
spec:
  containers:
  - name: busybox
    image: "k8s.gcr.io/busybox"
    volumeMounts:
    - name: demo-vol
      mountPath: "/demo"
    command: [ "sleep", "1000000" ]
  volumes:
    - name: demo-vol
      persistentVolumeClaim:
        claimName: demo-sc-pvc

Other references:

Kubernetes Storage on vSphere 101 – StorageClass - CormacHogan.comCormacHogan.com

community/container-storage-interface.md at 1a5277642cef37dd83273236ddf93bde67c342e1 · kubernetes/communityGitHub

Dynamic Volume ProvisioningKubernetes