# Experiment on Persistent Volume Access Mode This page mainly does some experiments on changing the access mode if PV or PVC is in use. The testing environment is on VMware TKG - Project Pacific. ## If Dynamic provisioning ### Create PVC and Pod ``` kind: PersistentVolumeClaim apiVersion: v1 metadata: name: daniel-pvc annotations: volume.beta.kubernetes.io/storage-class: gcstorage spec: accessModes: - ReadOnlyMany resources: requests: storage: 1Gi storageClassName: gcstorage ``` {% hint style="info" %} Make sure you have the storageclass available {% endhint %} ``` apiVersion: v1 kind: Pod metadata: name: daniel-pod spec: restartPolicy: Never containers: - name: hello image: "wcp-docker-ci.artifactory.eng.vmware.com/vmware/photon:1.0" # The script continously writes some text into the mounted persistent volume. This will ensure the pod is running and the persistent volume is accessible. command: ["/bin/sh", "-c", "echo 'hello' > /data/persistent/index.html && chmod o+rX /data /data/persistent/index.html && while true ; do sleep 2 ; done"] volumeMounts: - name: gc-persistent-storage mountPath: /data/persistent volumes: - name: gc-persistent-storage persistentVolumeClaim: claimName: daniel-pvc ``` Using `kubectl apply` to create PVC and Pod. After the creation, you should be able to see that pod is running, PVC is bound. ``` root@420f435c39833406727a02f08bc4a7b6 [ ~/daniel-tests ]# kubectl-gc get pvc daniel-pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE daniel-pvc Bound pvc-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f 1Gi RWO gcstorage 10m root@420f435c39833406727a02f08bc4a7b6 [ ~/daniel-tests ]# kubectl-gc get pod daniel-pod NAME READY STATUS RESTARTS AGE daniel-pod 1/1 Running 0 10m ``` ### Modify the access mode under PVC Modify the access mode under PVC to `ReadOnlyMany` ``` The PersistentVolumeClaim "daniel-pvc" is invalid: spec: Forbidden: is immutable after creation except resources.requests for bound claims ``` ### Modify the access mode under PV(which is dynamically created) Modify the access mode under PV to `ReadOnlyMany` ``` root@420f435c39833406727a02f08bc4a7b6 [ ~/daniel-tests ]# kubectl-gc edit pv pvc-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f persistentvolume/pvc-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f edited ``` #### Strange thing happens under the PVC spec, that spec and status are inconsistent: ``` spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: gcstorage volumeMode: Filesystem volumeName: pvc-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f status: accessModes: - ReadOnlyMany capacity: storage: 1Gi phase: Bound ``` #### However, CLI get returns the correct value: ``` root@420f435c39833406727a02f08bc4a7b6 [ ~/daniel-tests ]# kubectl-gc get pvc,pv NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/blog-content Bound pvc-386ef37c-94f3-4efa-9670-fde4e6f531a1 2Gi RWO gcstorage 3h7m persistentvolumeclaim/daniel-pvc Bound pvc-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f 1Gi ROX gcstorage 19m NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/pvc-386ef37c-94f3-4efa-9670-fde4e6f531a1 2Gi RWO Delete Bound default/blog-content gcstorage 3h7m persistentvolume/pvc-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f 1Gi ROX Delete Bound default/daniel-pvc gcstorage 19m ``` #### After changing the access mode to ReadOnlyMany: The Pod still can write data to the mounted volume #### Conclusion: * After changing the access mode under PV to `ReadOnlyMany`, the Pod runs fine and the volume is still writable * If delete the Pod and redeploy, the Pod could not be up. That is because VsphereVolume does not support `ReadOnlyMany` at this moment. [Details](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes) ``` 7s Warning FailedAttachVolume pod/daniel-pod AttachVolume.Attach failed for volume "pvc-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f" : rpc error: code = Internal desc = Validation for PublishVolume Request: {VolumeId:15aef1dd-8d27-4f46-a1da-0d0eb6a32fb9-bee5e54e-62ef-4abf-a40f-b6d3aa461d2f NodeId:cluster-md-0-55fddb665c-pmkhm VolumeCapability:mount: access_mode: Readonly:false Secrets:map[] VolumeContext:map[storage.kubernetes.io/csiProvisionerIdentity:1576057825043-8081-csi.vsphere.vmware.com type:vSphere CNS Block Volume] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} has failed. Error: rpc error: code = InvalidArgument desc = Volume capability not supported ``` ## If static provisioning ### Create PV, PVC and Pod ``` apiVersion: v1 kind: PersistentVolume metadata: name: daniel-staticpv-csi-driver annotations: pv.kubernetes.io/provisioned-by: csi.vsphere.vmware.com spec: storageClassName: daniel-static-provisioning-storageclass capacity: storage: 3Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Delete csi: driver: "csi.vsphere.vmware.com" volumeAttributes: type: "vSphere CNS Block Volume" volumeHandle: "95786abe-8f14-48fe-8f9d-c40094eaadc4-d842b247-5029-4f13-b8e8-999de471fb31" ``` {% hint style="info" %} volumeHandler above needs to reference to a valid PVC in Supervisor Cluster {% endhint %} ``` kind: PersistentVolumeClaim apiVersion: v1 metadata: name: daniel-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 3Gi storageClassName: daniel-static-provisioning-storageclass ``` ``` apiVersion: v1 kind: Pod metadata: name: daniel-pod spec: restartPolicy: Never containers: - name: hello image: "wcp-docker-ci.artifactory.eng.vmware.com/vmware/photon:1.0" # The script continously writes some text into the mounted persistent volume. This will ensure the pod is running and the persistent volume is accessible. command: ["/bin/sh", "-c", "echo 'hello' > /data/persistent/index.html && while true ; do sleep 2 ; done"] volumeMounts: - name: gc-persistent-storage mountPath: /data/persistent volumes: - name: gc-persistent-storage persistentVolumeClaim: claimName: daniel-pvc ``` ### Modify the access mode under PVC ``` root@420f435c39833406727a02f08bc4a7b6 [ ~/daniel-tests ]# kubectl-gc apply -f static-provision-pvc.yaml The PersistentVolumeClaim "daniel-pvc" is invalid: spec: Forbidden: is immutable after creation except resources.requests for bound claims ``` ### Modify the access mode under PV (Statically created) Modify the access mode to `ReadOnlyMany` ``` root@420f435c39833406727a02f08bc4a7b6 [ ~/daniel-tests ]# kubectl-gc apply -f static-provision-pv.yaml persistentvolume/daniel-staticpv-csi-driver configured root@420f435c39833406727a02f08bc4a7b6 [ ~/daniel-tests ]# kubectl-gc get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE daniel-staticpv-csi-driver 3Gi ROX Delete Bound default/daniel-pvc daniel-static-provisioning-storageclass 3h pvc-386ef37c-94f3-4efa-9670-fde4e6f531a1 2Gi RWO Delete Bound default/blog-content gcstorage 7h40m ``` However, the same inconsistency between spec and status happens as Dynamic Provisioning And Pod can still write data into the persistent volume after changing the access mode. #### Conclusion: The same conclusion as Dynamic provisinoing scenario