disk related help docs need update because some yaml files' path have been changed.

I deployed Lvm https://github.com/kubernetes-sigs/alibaba-cloud-csi-driver/blob/v1.14/deploy/lvm/lvm-plugin.yaml in k8s 1.16.0 .
csi-lvmplugin show this errors:
time="2019-10-30T10:38:25Z" level=info msg="CSI Driver Name: lvmplugin.csi.alibabacloud.com, , unix:///var/lib/kubelet/plugins/lvmplugin.csi.alibabacloud.com/csi.sock"
time="2019-10-30T10:38:25Z" level=info msg="CSI Driver Branch: 'v1.0.0', Version: 'v1.14.6-c1367ddd', Build time: '2019-10-11-20:12:48'\n"
time="2019-10-30T10:38:55Z" level=info msg="Use node id : "
E1030 10:38:55.039154 134718 driver.go:46] NodeID missing
I1030 10:38:55.039200 134718 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME
I1030 10:38:55.039239 134718 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME
I1030 10:38:55.039250 134718 driver.go:81] Enabling controller service capability: EXPAND_VOLUME
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x878e99]
goroutine 1 [running]:
github.com/kubernetes-csi/drivers/pkg/csi-common.(*CSIDriver).AddControllerServiceCapabilities(0x0, 0xc0002adddc, 0x3, 0x3)
/Users/junbao.kjb/work/project/go/pkg/mod/github.com/kubernetes-csi/[email protected]/pkg/csi-common/driver.go:85 +0x259
github.com/kubernetes-sigs/alibaba-cloud-csi-driver/pkg/lvm.NewDriver(0x0, 0x0, 0x7ffcf3b72ba9, 0x47, 0x3)
/Users/junbao.kjb/work/project/go/src/github.com/kubernetes-sigs/alibaba-cloud-csi-driver/pkg/lvm/lvm.go:58 +0x127
main.main()
/Users/junbao.kjb/work/project/go/src/github.com/kubernetes-sigs/alibaba-cloud-csi-driver/main.go:91 +0x6ca

Since the plugin only support the default folder of declaring plugin, a new configuration is required to configure the location of the custom kubelet data directory.

Per the CSI spec the name of a CSI driver must conform to the following requirements:

The name MUST follow domain name notation format (https://tools.ietf.org/html/rfc1035#section-2.3.1).

It SHOULD include the plugin's host company name and the plugin name, to minimize the possibility of collisions.

It MUST be 63 characters or less, beginning and ending with an alphanumeric character ([a-z0-9A-Z]) with dashes (-), dots (.), and alphanumerics between.

This field is REQUIRED.

The current names are:

• csi-diskplugin
• csi-nasplugin
• csi-ossplugin

Consider renaming to include the fully qualified domain name, maybe something like:

• diskplugin.csi.alibabacloud.com
• nasplugin.csi.alibabacloud.com
• ossplugin.csi.alibabacloud.com

To be clear the sepc says SHOULD include the plugin's host company name and the plugin name so this is not a requirement. But considering how generic these names are I strongly recommend it to prevent collisions with other CSI Drivers.

如题

fix code quality issues listed on :https://goreportcard.com/report/github.com/kubernetes-sigs/alibaba-cloud-csi-driver

Based on the oss.md file, there is no oss-attacher.yaml or oss-plugin.yaml files in folder deploy/oss/. Need a fix.

张家口阿里云服务器a区自建k8，版本1.12.3，3个master, 2个woker节点。
disk-plugin.yaml 文件已设置AK，但是创建pvc时显示pending.
另外高效云盘已格式化挂载到了master01上，是不是哪里需要指定volumnid？磁盘需要手动格式化吗？
多谢解答。

storageclass 设置如下：
# cat storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: csi-disk
provisioner: csi-diskplugin
parameters:
    zoneId: cn-zhangjiakou-a
    regionId: cn-zhangjiakou
    fsType: ext4
    type: cloud_efficiency
    readOnly: "false"
reclaimPolicy: Delete

csi pod已成功运行：
# kubectl get pod | grep -E 'csi|NAME'
NAME                                      READY   STATUS    RESTARTS   AGE
csi-diskplugin-47wtn                      2/2     Running   0          7h1m
csi-diskplugin-7wdvq                      2/2     Running   0          7h1m
csi-diskplugin-pl4qz                      2/2     Running   0          7h1m
csi-diskplugin-sn4cx                      2/2     Running   0          7h1m
csi-diskplugin-ztgfg                      2/2     Running   0          7h1m
csi-provisioner-attacher-0                2/2     Running   1          7h37m

pvc 状态为Pending：
# kubectl get sc,pv,pvc
NAME                                   PROVISIONER      AGE
storageclass.storage.k8s.io/csi-disk   csi-diskplugin   7h1m

NAME                             STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/disk-pvc   Pending                                      csi-disk       6h41m

pvc 没有错误日志：
# kubectl describe persistentvolumeclaim/disk-pvc
Name:          disk-pvc
Namespace:     default
StorageClass:  csi-disk
Status:        Pending
Volume:
Labels:        
Annotations:   control-plane.alpha.kubernetes.io/leader:
                 {"holderIdentity":"1b834f9a-2d87-11e9-ad9e-0a58c0a80006","leaseDurationSeconds":15,"acquireTime":"2019-02-10T23:52:08Z","renewTime":"2019-...
               volume.beta.kubernetes.io/storage-provisioner: csi-diskplugin
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
Events:
  Type       Reason                Age                       From                         Message
  ----       ------                ----                      ----                         -------
  Normal     ExternalProvisioning  2m10s (x2685 over 6h47m)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "csi-diskplugin" or manually created by system administrator
Mounted By:  

apply deploy/lvm/lvm-plugin.yaml , describe pod for ImagePullBackOff status :

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  34s                default-scheduler  Successfully assigned default/csi-lvm-plugin-bf4sf to master-1
  Normal   Pulling    33s                kubelet, master-1  Pulling image "registry.cn-hangzhou.aliyuncs.com/plugins/csi-node-driver-registrar:v1.0.1"
  Normal   Pulled     33s                kubelet, master-1  Successfully pulled image "registry.cn-hangzhou.aliyuncs.com/plugins/csi-node-driver-registrar:v1.0.1"
  Normal   Created    33s                kubelet, master-1  Created container driver-registrar
  Normal   Started    33s                kubelet, master-1  Started container driver-registrar
  Normal   Pulling    17s (x2 over 33s)  kubelet, master-1  Pulling image "registry.cn-hangzhou.aliyuncs.com/plugins/csi-lvmplugin:v1.13.2-e0e57163"
  Warning  Failed     17s (x2 over 33s)  kubelet, master-1  Failed to pull image "registry.cn-hangzhou.aliyuncs.com/plugins/csi-lvmplugin:v1.13.2-e0e57163": rpc error: code = Unknown desc = failed to resolve image "registry.cn-hangzhou.aliyuncs.com/plugins/csi-lvmplugin:v1.13.2-e0e57163": no available registry endpoint: registry.cn-hangzhou.aliyuncs.com/plugins/csi-lvmplugin:v1.13.2-e0e57163 not found
  Warning  Failed     17s (x2 over 33s)  kubelet, master-1  Error: ErrImagePull
  Normal   BackOff    3s (x3 over 32s)   kubelet, master-1  Back-off pulling image "registry.cn-hangzhou.aliyuncs.com/plugins/csi-lvmplugin:v1.13.2-e0e57163"
  Warning  Failed     3s (x3 over 32s)   kubelet, master-1  Error: ImagePullBackOff

What is the correct image tag ?

As csi-plugin is a cloud native app, it is better for it to emit logs to stdout.
Need command arguments to let the plugin log to stdout instead of file.

unable to recognize "csi-plugin.yaml": no matches for kind "CSIDriver" in version "storage.k8s.io/v1beta"

版本：
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.6-aliyun.1", GitCommit:"8cb561c", GitTreeState:"", BuildDate:"2019-04-22T11:34:20Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

Events: Successfully assigned zk-0 to cn-shanghai.i-uf62ixdy0e67uu23f7vp AttachVolume.Attach succeeded for volume "pvc-91bd899c-9c8e-11e8-a639-00163e1a81be" MountVolume.SetUp succeeded for volume "default-token-cbhb4" MountVolume.SetUp failed for volume "pvc-91bd899c-9c8e-11e8-a639-00163e1a81be" : rpc error: code = DeadlineExceeded desc = context deadline exceeded Unable to mount volumes for pod "zk-0_dev(746039aa-9c91-11e8-a639-00163e1a81be)": timeout expired waiting for volumes to attach or mount for pod "dev"/"zk-0". list of unmounted volumes=[datadir]. list of unattached volumes=[datadir default-token-cbhb4] MountVolume.SetUp failed for volume "pvc-91bd899c-9c8e-11e8-a639-00163e1a81be" : rpc error: code = Unknown desc = NodePublishVolume: volume is mounting: /var/lib/kubelet/pods/746039aa-9c91-11e8-a639-00163e1a81be/volumes/kubernetes.io~csi/pvc-91bd899c-9c8e-11e8-a639-00163e1a81be/mount

any plan for this?

Lvm pv 缺少 nodeAffinity 导致 pod 重启后不能跟之前的pv 绑定，跑到其它机器上了导致pod起不来。
请参考 https://github.com/wavezhang/k8s-csi-lvm
中的https://github.com/wavezhang/k8s-csi-lvm/blob/master/pkg/lvm/util.go 和
https://github.com/wavezhang/k8s-csi-lvm/blob/master/pkg/lvm/nodeserver.go

主要代码：

func generateNodeAffinity(node *v1.Node) (*v1.VolumeNodeAffinity, error) {
	if node.Labels == nil {
		return nil, fmt.Errorf("Node does not have labels")
	}
	nodeValue, found := node.Labels[NodeLabelKey]
	if !found {
		return nil, fmt.Errorf("Node does not have expected label %s", NodeLabelKey)
	}

	return &v1.VolumeNodeAffinity{
		Required: &v1.NodeSelector{
			NodeSelectorTerms: []v1.NodeSelectorTerm{
				{
					MatchExpressions: []v1.NodeSelectorRequirement{
						{
							Key:      NodeLabelKey,
							Operator: v1.NodeSelectorOpIn,
							Values:   []string{nodeValue},
						},
					},
				},
			},
		},
	}, nil
}

func (ns *nodeServer) createVolume(ctx context.Context, volumeId string) (*v1.PersistentVolume, error) {
。。。。。
nodeAffinityAnn, err := generateNodeAffinity(node)
	if err != nil {
		return nil, status.Errorf(codes.Internal, fmt.Sprintf("Failed to generate node affinity annotations for %v: %v", node.GetName(), err))
	}
pv.Spec.NodeAffinity = nodeAffinityAnn
	pv.Annotations[lvmNodeAnnKey] = node.GetName()
	return updatePV(ns.client, pv)
。。。。。
}

正确的pv 格式 如下，这样pod 重启后才不会漂移到其它机器上。才能和之前的pv 绑定。

apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    lvm/node: 10.10.3.148
    pv.kubernetes.io/provisioned-by: csi-lvmplugin-ssd
  creationTimestamp: 2019-05-23T14:50:34Z
  name: pvc-1ebba74c-7d6a-11e9-890d-ee86db64b4a1
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 100Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: testfs1
    namespace: ike
    resourceVersion: "56365033"
    uid: 1ebba74c-7d6a-11e9-890d-ee86db64b4a1
  csi:
    driver: csi-lvmplugin-ssd
    fsType: ext4
    volumeAttributes:
      fstype: ext4
      storage.kubernetes.io/csiProvisionerIdentity: 1558518299074-8081-csi-lvmplugin-ssd
    volumeHandle: pvc-1ebba74c-7d6a-11e9-890d-ee86db64b4a1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - 10.10.3.148
  persistentVolumeReclaimPolicy: Delete
  storageClassName: lvm-ssd
  volumeMode: Filesystem

我的 sc 设置的是

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: csi-lvm
  selfLink: /apis/storage.k8s.io/v1/storageclasses/csi-lvm
  uid: 1d600dbb-b36a-4374-9f1d-a752e78725c6
  resourceVersion: '10502032'
  creationTimestamp: '2019-11-08T07:50:31Z'
provisioner: lvmplugin.csi.alibabacloud.com
parameters:
  fsType: ext4
  vgName: k8s
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate

当把pod 删除后 ，手动删除对应的 pvc ，这时pv自动删除了。但是通过 lvscan 命令查看发现 对应刚才已经删除的 pv 中使用的逻辑卷 LV 并没有删除。这样会留下大量垃圾LV ，需要手动删除这些LV么？

Local Volume Provisioner features:

support auto create local volume;

support alicloud local disk and hostpath type;

pv should be available after pvc deleted;

目前阿里云盘创建时可以指定ResourceGroupId字段，如果ram权限配置有不同资源组时，创建云盘时必须指定ResourceGroupId，否则报错：

而csi-plugin目前创建云盘时不支持传入ResourceGroupId字段，导致创建云盘始终失败：

I0710 08:43:02.286344       1 controller.go:544] CreateVolumeRequest {Name:pvc-34e84c6c-94cc-45a1-b5f5-f2f8f23c2021 CapacityRange:required_bytes:26843545600  VolumeCapabilities:[mount:<fs_type:"ext4" > access_mode:<mode:SINGLE_NODE_WRITER > ] Parameters:map[readOnly:false regionId:cn-shenzhen type:cloud_ssd zoneId:cn-shenzhen-e fsType:ext4] Secrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:<nil> XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0710 08:43:02.286495       1 controller.go:188] GRPC call: /csi.v1.Controller/CreateVolume
I0710 08:43:02.286563       1 controller.go:189] GRPC request: {"capacity_range":{"required_bytes":26843545600},"name":"pvc-34e84c6c-94cc-45a1-b5f5-f2f8f23c2021","parameters":{"fsType":"ext4","readOnly":"false","regionId":"cn-shenzhen","type":"cloud_ssd","zoneId":"cn-shenzhen-e"},"volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]}
I0710 08:43:02.393139       1 controller.go:191] GRPC response: {}
I0710 08:43:02.393510       1 controller.go:192] GRPC error: rpc error: code = Internal desc = SDK.ServerError
ErrorCode: Forbidden.RAM
Recommend: 
RequestId: F53FBFB4-E4CA-4D41-80C2-76450FF817CC

所以请求csi-plugin也支持在storageclass.yaml中支持指定ResourceGroupId，以便能成功创建云盘~

PS：资源组相关的文档

Update nas deploy docs

Hello，
请问如何挂载已有的云盘呢？像指定云盘volumeId？我查看了现在的文档和example没有看到有提到相关的信息。

Nas filesystem volume's default delete policy should be false

One pod is using OSS volume.
The mountpoint will get "transport endpoint is not connected" error, if restart the ossplugin.

The reason is oss volume use fuse to connect with server, which is in user space. Restart ossplugin will kill all process belongs to the pod.

Using "systemd-run --scope -- " is also useless.

Issues is tracking this:
kubernetes/kubernetes#70013
kubernetes/kubernetes#7890

NAS Provisioner will work in two modes:

1. Provision PV with subPath using exist nas mount server(supprted);
2. Provision PV with fileSystem(need to support);

Three types of nas fs should be supported: Capacity Type, Performance Type, Extreme Type.

按照git上的文档安装了下，csi-provisioner-attacher 启动不起来，查看日志 提示

Error from server (BadRequest): a container name must be specified for pod csi-provisioner-attacher-0, choose one of: [csi-provisioner csi-attacher]

用 describe 命令 提示
Warning FailedScheduling 3m (x1261 over 1h) default-scheduler 0/3 nodes are available: 3 node(s) didn't match node selector.

修改了 storageclass.yaml 里的
zoneId: cn-shenzhen-a
regionId: cn-shenzhen
阿里云服务器属于 华南

csi-lvm-plugin should support nodeAffinity in dynamic volume.

Currently, the dynamic pv not contains nodeAffinity info, which is useful for some usecases, and must use nodeSelector to schedule Pod to one node.

add go report card in README file

add code qualit checking scripts: fmt, golint, govet, ut

ESSD云盘有3个性能级别，不知在YAML文件中如何指定？

pwd

/tmp/csi-plugin/build

sh build-disk.sh

main.go:27:2: cannot find package "github.com/AliyunContainerService/csi-plugin/pkg/disk" in any of:
/usr/local/go/src/github.com/AliyunContainerService/csi-plugin/pkg/disk (from $GOROOT)
/root/go/src/github.com/AliyunContainerService/csi-plugin/pkg/disk (from $GOPATH)
main.go:28:2: cannot find package "github.com/AliyunContainerService/csi-plugin/pkg/lvm" in any of:
/usr/local/go/src/github.com/AliyunContainerService/csi-plugin/pkg/lvm (from $GOROOT)
/root/go/src/github.com/AliyunContainerService/csi-plugin/pkg/lvm (from $GOPATH)
main.go:29:2: cannot find package "github.com/AliyunContainerService/csi-plugin/pkg/nas" in any of:
/usr/local/go/src/github.com/AliyunContainerService/csi-plugin/pkg/nas (from $GOROOT)
/root/go/src/github.com/AliyunContainerService/csi-plugin/pkg/nas (from $GOPATH)
main.go:30:2: cannot find package "github.com/AliyunContainerService/csi-plugin/pkg/oss" in any of:
/usr/local/go/src/github.com/AliyunContainerService/csi-plugin/pkg/oss (from $GOROOT)
/root/go/src/github.com/AliyunContainerService/csi-plugin/pkg/oss (from $GOPATH)
main.go:31:2: cannot find package "github.com/Sirupsen/logrus" in any of:
/usr/local/go/src/github.com/Sirupsen/logrus (from $GOROOT)
/root/go/src/github.com/Sirupsen/logrus (from $GOPATH)

support to create and delete nas filesystem and mountTarget dynamic when create or delete a nas volume

I noticed that there are some other images used in folder deploy/disk. They are

• registry.cn-hangzhou.aliyuncs.com/plugins/driver-registrar:v0.2.0
• registry.cn-hangzhou.aliyuncs.com/plugins/csi-attacher:v0.2.0
• registry.cn-hangzhou.aliyuncs.com/plugins/csi-provisioner:v0.2.0

Are there any source code for them I could find in github?

My host is on cn-beijing-f but the volume is created at cn-beijing-b.

func (cs *controllerServer) CreateVolume(ctx context.Context, req *csi.CreateVolumeRequest) (*csi.CreateVolumeResponse, error) {

The input param req contains zone info cn-beijing-b. Where is cn-beijing-b from?

deploy/oss中没有oss-attacher.yaml，是文档没有更新吗？

我在 kubelet 开启了
--feature-gates=ExpandCSIVolumes=true
pv，pvc更新正常
resizer

I0929 07:39:01.466998       1 controller.go:195] Started PVC processing "kube-system/disk-pvc"
I0929 07:39:01.471902       1 connection.go:180] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I0929 07:39:01.471982       1 event.go:209] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"kube-system", Name:"disk-pvc", UID:"b062e9c3-9f07-402a-95cb-69fc606d69ba", APIVersion:"v1", ResourceVersion:"7602221", FieldPath:""}): type: 'Normal' reason: 'Resizing' External resizer is resizing volume pv-disk-b062e9c3-9f07-402a-95cb-69fc606d69ba
I0929 07:39:01.471951       1 connection.go:181] GRPC request: {"capacity_range":{"required_bytes":26843545600},"volume_id":"d-bp110u5v2hhsep6t2t2c"}
I0929 07:39:04.225417       1 leaderelection.go:258] successfully renewed lease kube-system/external-resizer-diskplugin-csi-alibabacloud-com
I0929 07:39:04.480906       1 connection.go:183] GRPC response: {"capacity_bytes":26843545600,"node_expansion_required":true}
I0929 07:39:04.481413       1 connection.go:184] GRPC error: <nil>
I0929 07:39:04.481425       1 controller.go:329] Resize volume succeeded for volume "pv-disk-b062e9c3-9f07-402a-95cb-69fc606d69ba", start to update PV's capacity
I0929 07:39:04.494776       1 controller.go:335] Update capacity of PV "pv-disk-b062e9c3-9f07-402a-95cb-69fc606d69ba" to 25Gi succeeded
I0929 07:39:04.498708       1 controller.go:385] Mark PVC "kube-system/disk-pvc" as file system resize required
I0929 07:39:04.498813       1 event.go:209] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"kube-system", Name:"disk-pvc", UID:"b062e9c3-9f07-402a-95cb-69fc606d69ba", APIVersion:"v1", ResourceVersion:"7602221", FieldPath:""}): type: 'Normal' reason: 'FileSystemResizeRequired' Require file system resize of volume on node

csi-diskplugin log

29 07:34:09.507678       8 controllerserver-default.go:62] Using default ControllerGetCapabilities
I0929 07:34:09.507684       8 utils.go:102] GRPC response: capabilities:<rpc:<type:CREATE_DELETE_VOLUME > > capabilities:<rpc:<type:PUBLISH_UNPUBLISH_VOLUME > > capabilities:<rpc:<type:CREATE_DELETE_SNAPSHOT > > capabilities:<rpc:<type:LIST_SNAPSHOTS > > capabilities:<rpc:<type:EXPAND_VOLUME > >
I0929 07:39:01.473331       8 utils.go:96] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I0929 07:39:01.473358       8 utils.go:97] GRPC request: volume_id:"d-bp110u5v2hhsep6t2t2c" capacity_range:<required_bytes:26843545600 >
time="2019-09-29T07:39:01Z" level=info msg="ControllerExpandVolume:: Starting expand disk with: volume_id:\"d-bp110u5v2hhsep6t2t2c\" capacity_range:<required_bytes:26843545600 > "
time="2019-09-29T07:39:04Z" level=info msg="ControllerExpandVolume:: Success to resize volume: d-bp110u5v2hhsep6t2t2c from 20G to 25G"
I0929 07:39:04.480534       8 utils.go:102] GRPC response: capacity_bytes:26843545600 node_expansion_required:true

pv-disk-b062e9c3-9f07-402a-95cb-69fc606d69ba   25Gi       RWO            Delete           Bound    kube-system/disk-pvc

但是文件系统还是没有更新
/dev/vdd 20G 45M 20G 1% /data

1. k8s, single worker node, version 1.11.0
2. cis, version v0.3.0
3. apply example Yaml below.
4. get into nginx pod, and create a file in /data by touch test
5. delete current nginx pod
6. get into the new nginx pod, the /data folder is wrongly mounted.

Root cause: the disk is attached from the machine.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: disk-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 25Gi
  storageClassName: csi-disk
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment1
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
        volumeMounts:
          - name: disk-pvc
            mountPath: "/data"
      volumes:
        - name: disk-pvc
          persistentVolumeClaim:
            claimName: disk-pvc

This is my storageclass.yaml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: csi-disk
provisioner: csi-diskplugin
parameters:
    zoneId: cn-shanghai-b
    regionId: cn-shanghai
    fsType: ext4
    type: cloud_ssd
    readOnly: "false"
reclaimPolicy: Delete

I set reclaimPolicy to Delete, that means a volume will be automatically deleted when I delete the corresponding pvc, but after I delete a pvc, I got this message:

 rpc error: code = Unknown desc = Delete disk 3 times, but error: Aliyun API Error: RequestId: 526A2F7D-BF04-4F9C-8BED-DCA19603EDDC Status Code: 403 Code: IncorrectDiskStatus Message: The current disk status does not support this operation.

Any help would be greatly appreciated.

update nas docs with creating 'filesystem' type nas volume

k8s verison: 1.11.0
csi plugin versions: v0.3.0
There are 3+ pvs mounted on a single VM.
When I try to remove all the PVs at the same time, at least 1 PV can't be removed.

Root cause:

1. NodeUnpublishVolume returns error. it is

 time="2018-12-27T08:24:27Z" level=error msg="NodeUnpublishVolume: detach error %!(EXTRA string=d-uf62khufqdz7w8hybmu9, string=Aliyun API Error: RequestId: 885218BD-391A-4C6B-BA40-0FCA3EB6019E Status Code: 403 Code: InvalidOperation.Conflict Message: null)"
E1227 08:24:27.715595    2106 utils.go:100] GRPC error: rpc error: code = Internal desc = Detach error:Aliyun API Error: RequestId: 885218BD-391A-4C6B-BA40-0FCA3EB6019E Status Code: 403 Code: InvalidOperation.Conflict Message: null

Though this function returns an error, Kubelete did NO retry to called. And Provisioner calls DeleteVolume directly. This will always fails because the volume is not detached from the machine.

2. Why error InvalidOperation.Conflict happens?
   Because Alicloud API call has rate limeter.

add disk examples to use a resourcegroup or byok and update all image urls from xxx/plugins/xxx to xxx/acs/xxx

CPFS Provisioner will work in 2 mdoes:

1. provision PV with subpath from exist CPFS;
2. provision PV with fileSystem;

cat storageclass_hdd.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: alicloud-csi-disk-efficiency
annotations:
storageclass.kubernetes.io/is-default-class: true
provisioner: diskplugin.csi.alibabacloud.com
parameters:
zoneId: cn-hangzhou-g
regionId: cn-hangzhou
fsType: ext4
type: cloud_efficiency
readOnly: "false"
reclaimPolicy: Retain
allowVolumeExpansion: true

cat pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: disk-pvc-efficiency
spec:
accessModes:

• ReadWriteOnce
  resources:
  requests:
  storage: 20Gi
  storageClassName: alicloud-csi-disk-efficiency

kubectl get pvc

NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
disk-pvc-efficiency Bound pvc-d53ed818-8840-11e9-a463-00163e116758 20Gi RWO alicloud-csi-disk-efficiency 21m

当我将 pvc 改成 30 GI 时候，没有发现扩容：

cat pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: disk-pvc-efficiency
spec:
accessModes:

• ReadWriteOnce
  resources:
  requests:
  storage: 30Gi
  storageClassName: alicloud-csi-disk-efficiency

kubectl get pvc

NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
disk-pvc-efficiency Bound pvc-d53ed818-8840-11e9-a463-00163e116758 20Gi RWO alicloud-csi-disk-efficiency 21m

Kubernetes 1.12 introduced some breaking changes, it makes current release (v0.2.0) is not working at all.

From PR Add CSI action required to 1.12 release notes

• In anticipation of CSI 1.0 in the next release, Kubernetes 1.12 calls the CSI NodeGetInfo RPC instead of NodeGetId RPC. Ensure your CSI Driver implements NodeGetInfo(...) before upgrading to 1.12. @saad-ali
• Kubernetes 1.12 also enables Kubelet device plugin registration feature by default. Before upgrading to 1.12, ensure the driver-registrar CSI sidecar container for your CSI driver is configured to handle plugin registration (set the --kubelet-registration-path parameter on driver-registrar to expose a new unix domain socket to handle Kubelet Plugin Registration).

Any and all help is much appreciated.

1. csi-plugin spport add aliyun ebs tag;
2. tag can be specified by users.

创建pvc指定datasource为volumeSnapshot

modify subpath mount dir from xxx/-delete to xxx-delete

add PR status check jobs to verify fmt, golint, go vet and unit tests

1、$ cd build && sh build-oss.sh
2、# kubectl create -f ./deploy/csidriver.yaml --validate=false
3、# kubectl create -f ./deploy/oss/oss-plugin.yaml

4、查看CSIDriver状态：
[root@vm-zdyh3linux-1 deploy]# kubectl describe CSIDriver ossplugin.csi.alibabacloud.com
Name: ossplugin.csi.alibabacloud.com
Namespace:
Labels:
Annotations:
API Version: csi.storage.k8s.io/v1alpha1
Kind: CSIDriver
Metadata:
Creation Timestamp: 2019-08-08T02:23:17Z
Generation: 1
Resource Version: 393449
Self Link: /apis/csi.storage.k8s.io/v1alpha1/csidrivers/ossplugin.csi.alibabacloud.com
UID: 7bba4fe4-b983-11e9-ad8e-525400ccffd1
Spec:
Attach Required: false
Events:
[root@vm-zdyh3linux-1 deploy]#

5、修改deploy.yaml中pv的bucket url akid aksecret(这几项都是从页面中获取)，otherOpts没有改；

6、kubectl create -f deply.yaml
pv和pvc绑定成功，但是pod的状态出错：timeout expired waiting for volumes to attach or mount for pod
[root@vm-zdyh3linux-1 deploy]# kubectl describe pod deployment-oss-57d5695645-7gtcn
Name: deployment-oss-57d5695645-7gtcn
Namespace: default
Priority: 0
PriorityClassName:
Node: vm-zdyh3linux-1/192.168.0.181
Start Time: Thu, 08 Aug 2019 10:57:33 +0800
Labels: app=nginx
pod-template-hash=1381251201
Annotations:
Status: Pending
IP:
Controlled By: ReplicaSet/deployment-oss-57d5695645
Containers:
nginx:
Container ID:
Image: nginx:1.7.9
Image ID:
Port: 80/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment:
Mounts:
/data from oss-pvc (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zrxm4 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
oss-pvc:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: oss-pvc
ReadOnly: false
default-token-zrxm4:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zrxm4
Optional: false
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message

Warning FailedScheduling 9m (x2 over 9m) default-scheduler pod has unbound PersistentVolumeClaims (repeated 3 times)
Warning FailedScheduling 9m default-scheduler AssumePod failed: pod 44ac908f-b988-11e9-ad8e-525400ccffd1 is in the cache, so can't be assumed
Normal Scheduled 9m default-scheduler Successfully assigned default/deployment-oss-57d5695645-7gtcn to vm-zdyh3linux-1
Warning FailedMount 1m (x4 over 7m) kubelet, vm-zdyh3linux-1 Unable to mount volumes for pod "deployment-oss-57d5695645-7gtcn_default(44ac908f-b988-11e9-ad8e-525400ccffd1)": timeout expired waiting for volumes to attach or mount for pod "default"/"deployment-oss-57d5695645-7gtcn". list of unmounted volumes=[oss-pvc]. list of unattached volumes=[oss-pvc default-token-zrxm4]
Warning FailedAttachVolume 29s (x5 over 9m) attachdetach-controller AttachVolume.Attach failed for volume "oss-csi-pv" : attachment timeout for volume data-id
[root@vm-zdyh3linux-1 deploy]#

add option to support delele policy in filesysyem type nas volume

I have deploy the CSI plugin in my local kubernetes V1.13, and enable FeatureGate about CSI in kubelet and api-server, and I have set my AK account and secret and bucket name in ./deploy/oss/deploy.yaml
when describe pod nginx it says driver name not found in the list of registered CSI drivers as the picture says blow

can you help me fix it ?
thank you