kubernetes-csi / docs Goto Github PK
View Code? Open in Web Editor NEWDocumentation for CSI integration with Kubernetes
License: Apache License 2.0
Documentation for CSI integration with Kubernetes
License: Apache License 2.0
As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.
The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".
Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)
Thanks so much, let me know if you have any questions.
(This issue was generated from a tool, apologies for any weirdness.)
[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md
The recommended approach is to deploy the controller in a StatefulSet and the node plugin in a DaemonSet. The example should follow that recommendation, even though it is not strictly needed for a driver that only works in a single node.
It does fix one specific issue: directly creating a pod as in the current example can fail due to a race condition (kubernetes/kubernetes#67882).
The URL for CSINodeInfo is outdated here:
https://kubernetes-csi.github.io/docs/Setup.html#enabling-csinodeinfo
It was:
https://raw.githubusercontent.com/kubernetes/csi-api/master/pkg/crd/testdata/csinodeinfo.yaml
Not it becomes:
https://raw.githubusercontent.com/kubernetes/csi-api/master/pkg/crd/manifests/csinodeinfo.yaml
Also, the same issue for CSIDriver CRD
I need to add documentation to https://kubernetes-csi.github.io/docs/sidecar-containers.html
Background:
A new CSI Metrics Library was added to csi-lib-utils
in and is part of v0.7.0 release. This library can be used to automatically generate Prometheus metrics for all CSI operations including total count, error count, and call latency. This library was integrated in to the following CSI Sidecar containers:
New flags “--metrics-address” or “--metrics-path” are now part of all 4 of those sidecars. Driver deployments should set those flags to ensure the metrics are being emitted.
We should add documentation on how to create a volume snapshot with static binding.
Create this issue to add the CSI driver for OpenSDS, an open source project under Linux Foundation (https://www.opensds.io/), to the drivers table. For more information, please visit https://github.com/opensds/nbp/releases (beta release) and https://github.com/opensds/nbp/tree/master/csi.
Hi,
As the instructions written at https://kubernetes-csi.github.io/docs/Drivers.html I would like to add our CSI plugin for QingCloud block storage to the list of production drivers.
The driver repo is here: https://github.com/yunify/qingcloud-csi
Name: QingCloud
Status: v0.2.0
Description: A Container Storage Interface (CSI) Plugin for QingCloud Block Storage
Please let me know if you need more information.
Thanks
Updating docs is a two step process:
book/*
docs/*
We should add a Makefile at the root that makes it dead simple to run the command:
make all <-- generate all files
make clean <-- remove temp files
make clobber <-- remove all generated files
Hi,
As per the instructions written at https://kubernetes-csi.github.io/docs/Drivers.html I would like to add our CSI plugin for DigitalOcean block storage to the list of production drivers.
The driver repo is here: https://github.com/digitalocean/csi-digitalocean
Name: DigitalOcean Block Storage
Status: Alpha
Description: A Container Storage Interface (CSI) Driver for DigitalOcean Block Storage
We're working on releasing the initial version v0.0.1
. Until then it's marked as alpha
. Please let me know if you need more information.
Thanks
We probably don't have good documentation about when to use StatefulSet and when to use Deployment for the controller. See container-storage-interface/spec#364
Leadership election needs to be covered, too.
[ ]
The version table for https://kubernetes-csi.github.io/docs/livenessprobe.html currently looks like this:
Latest stable release | Branch | Min CSI Version | Max CSI Version | Container Image | Min K8s Version | Max K8s Version | --|--|--|--|--|-- livenessprobe v2.0.0 | release-2.0 | v1.0.0 | quay.io/k8scsi/livenessprobe:v2.0.0 | v1.13 | - livenessprobe v1.1.0 | release-1.1 | v1.0.0 | quay.io/k8scsi/livenessprobe:v1.1.0 | v1.13 | - Unsupported. | No 0.x branch. | v0.3.0 | quay.io/k8scsi/livenessprobe:v0.4.1 | v1.10 | v1.16
Looks like it may have broken in #290
CC @msau42
On the https://kubernetes-csi.github.io/docs/drivers.html page, GCE PD CSI Driver should list "Expansion" under "Other Features"
Hi there, we have finished our CSI driver now .
vendor: XSKY
description: A Container Storage Interface (CSI) Driver for XSKY distributed block storage (X-EBS)
version: Beta
homepage: https://www.xsky.com/en/
please help adding to table Production Drivers
thanks
Keeping a table of production plugins and the features they implement is handy, but as the project has grown, along with the number of plugins and features; this simple table has become a bit difficult to view and even more difficult to update and keep accurate.
I don't have a solution to reformat what's in place, but I wonder if instead of trying to keep this up to date for features, replace the features columns with a link to the plugins documentation? Said link could be a plugin doc (we could provide a template for plugin authors to use) and we could host that in this repo under sub-sections (sub-section for each plugin).
The downside of this is that users like to have the matrix in front of them to compare things, so this doesn't help with that. It does however clean things up in terms of listing available plugins and their features; it provides the information, just no easy way to compare plugins.
Opening this issue to propose the template idea or to solicit input for a better suggestion if someone has one.
Add CSI driver for ChubaoFS. ChubaoFS is a distributed file system for cloud native applications. https://www.chubao.io/
Move it out to a top level page: https://kubernetes-csi.github.io/docs/sidecar-containers.html
It's hard to keep track of changes between releases and compose correct CHANGELOG.
Proposal:
Add PR template to all kubernetes-csi repos. Template of the template: kubernetes-csi/external-attacher#139. It's slightly trimmed version of kubernetes/kubernetes template with removed links to Kubernetes specific docs.
Enable release-notes bot in prow that will nag people to fill ```release-note
(same as in kubernetes/kubernetes).
PR reviewers/approvers must ensure that ```release-note
is filled in PRs where is makes sense.
Use release-notes tool to generate CHANGELOG from ```release-note
, ```release-note Action required:
and /kind {feature|bug}
in PRs. See Kubernetes 1.11 - we won't get SIG sections, but the rest should be similar. Some manual editing is still required (deprecations?).
@kubernetes-csi/csi-misc
The documentation for the node driver registrar links to the Device Plugin registration instructions. While we were reimplementing the registration API, it turns out this is incorrect. Insofar as I can tell, the node driver registrar uses the plugin watcher mechanism to register a CSI Driver and not the device plugin endpoint: https://github.com/kubernetes/kubelet/blob/088ee84ea259bf9c445109ea75b2938dd39d2074/pkg/apis/pluginregistration/v1/api.proto#L57
I would open a PR, but I am unclear how this should be documented because there isn't much user facing (as opposed to developer) documentation on the plugin watcher. Any direction would be welcome and then I can open a PR
For new developers it would be useful to document how they can build and test CSI. This could include:
Please update in the CSI Drivers page (https://kubernetes-csi.github.io/docs/Drivers.html)
ScaleIO | v0.1.0 | A Container Storage Interface (CSI) Storage Plugin for DellEMC ScaleIO
to
ScaleIO | v0.1.0 (Tech Preview)| A Container Storage Interface (CSI) Storage Plugin for DellEMC ScaleIO
This is temporary, until we make the new VxFlex OS CSI driver available.
Regards,
Eyal S.
VxFlex OS Dell EMC
/assign @gnufied
CSI Proxy Alpha is complete. There are changes in Kubernetes 1.18 that enable its use.
We should add a new page under https://kubernetes-csi.github.io/docs/features.html for "Windows Support" to document CSI Windows Support (e.g. what do Kubernetes distributors need to do? what do CSI driver vendors need to do, etc.)
Greetings,
We would like to add our CSI plugin for NexentaEdge File,Block and Object storage to the list of production drivers.
The driver repo is here: https://github.com/Nexenta/nexentaedge-csi-driver
Name: NexentaEdge https://nexentaedge.io/docs/kubernetes-csi.html
Status: v0.2.0
Description: A Container Storage Interface (CSI) Plugin for NexentaEdge Scale-Out, Multi-Cloud S3/NFS storage services
Thanks
If a CSI driver implements NodeStageVolume
, then it must have a volumeMount
entry which ensures that /var/lib/kubelet/plugins/kubernetes.io/csi
(or is it /var/lib/kubelet/plugins/kubernetes.io/csi/pv/
?) is shared with the host.
Otherwise the driver is not going to see the staging directory that kubelet creates for it, like /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4863bcae-4b2f-11e9-9bd3-deadbeef0100/globalmount
.
In practice, drivers like gcepd have solved this instead by creating the staging directory themselves if needed:
https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/5d62c770f3e331e4236ebe42b74321f007f8c1e7/pkg/gce-pd-csi-driver/node.go#L201-L208
We should:
NodeStageVolume
)Instead of redirecting the reader to a repo
KubeletPluginsWatcher
is enabled by default in Kubernetes v1.12, the CSI plugin registration mechanism has been changed and Kubelet registers CSI plugins through kubelet-registration-path
.KubeletPluginsWatcher
feature gate on Kubelet. In my opinion, we could add this case to the chapter of troubleshooting.KubeletPluginsWatcher
feature gate before deploying CSI plugin based on CSI spec v0.2.0, can we deploy and use CSI plugins based on v0.3.0 and v0.2.0 in a Kubernetes v1.12 cluster simultaneously?The table from
https://github.com/kubernetes-csi/docs/blob/master/book/src/drivers.md#production-drivers
has column Supported Access Modes
which has value like Read/Write Single Pod
or Read/Write Multiple Pods
According to Kubernetes document https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes
The access modes are:
ReadWriteOnce – the volume can be mounted as read-write by a single node
ReadOnlyMany – the volume can be mounted read-only by many nodes
ReadWriteMany – the volume can be mounted as read-write by many nodes
My point is that Read/Write Multiple Pods is not equal to read-write by many nodes
case 1 : what if Pods in the same node?
case 2: what if Pods in different node?
Also take a look at csi specification https://github.com/container-storage-interface/spec/blob/4731db0e0bc53238b93850f43ab05d9355df0fd9/csi.proto#L365
it also define AccessMode base on SINGLE_NODE/MULTI_NODE.
It is necessary to clarify Access Mode.
I wrote a csi driver for synology NAS(https://github.com/jparklab/synology-csi), and would like to share it with others.
Can you please add it to the sample driver list?
Name | Status | More Information |
---|---|---|
Synology NAS | v1.0.0 | A CSI Plugin for synology NAS(https://www.synology.com/en-us) |
The example page should be redone to point to new location of hostpath and hostpath doc https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/README.md.
The link to Hostpath driver here is not working https://kubernetes-csi.github.io/docs/deploying.html
We should have a best practices page outlining suggestions for deploying CSI drivers that document things like:
We can also extend that by providing a linter that vendors can use on their deployment specs. The linter could also detect things like:
@kubernetes-csi/csi-misc
Kubernetes CSI supports topology and getting volume stats.
We can add topology awareness and getting volume stats to the drivers feature table.
CSI Provisioner Parameters were deprecated and replaced in v1.0.0: kubernetes-csi/external-provisioner#178
The documentation should reflect the new way of doing things. Being very careful to make sure that its clear that the name spaced parameters will NOT be passed through to CreateVolume
.
/assign
/cc @msau42
Setting --csi-address=$(CSI_ADDRESS)
is unnecessarily complex. The extra env variable isn't needed.
Can it be invoked through kube-apiserver ? If so, how to invoke ? Or any other ways to get Volume List information ?
It can mirror the Kubernetes policy.
Define policy on:
This may also tie into defining our sidecar versioning scheme and what warrants a major, minor and patch version bump.
Updating docs is a two step process:
book/*
docs/*
To ensure that some one doesn't accidentally only update book/*
or only modifies the generated files, we should add a new PR check (via travis maybe?) to verify that generated files do not differ from the generated files in the PR.
Link to page: https://kubernetes-csi.github.io/docs/Example.html
I've been trying to develop my own CSI driver, but unfortunately I can't even seem to get the example in the docs running with minikube. I have tried it on both a macOS and linux host with the kubeadm bootstraps and the kvm2, virtualbox, and hyperkit drivers for minikube.
I am consistently getting the following error:
Name: csi-pod
Namespace: default
Node: minikube/192.168.64.3
Start Time: Fri, 03 Aug 2018 11:13:58 -0700
Labels: app=hostpath-driver
Annotations: <none>
Status: Running
IP: 172.17.0.4
Containers:
external-provisioner:
Container ID: docker://1e8f87f4d3a4eb3c603afc6304c6040d746e7fe1b698f1339dce3899dbd467b5
Image: quay.io/k8scsi/csi-provisioner:v0.2.1
Image ID: docker-pullable://quay.io/k8scsi/csi-provisioner@sha256:fd4ed32315e846b6654f97c95b373da001fd9638cd5935c20a5bf9f5889e8602
Port: <none>
Host Port: <none>
Args:
--v=5
--provisioner=csi-hostpath
--csi-address=/csi/csi.sock
State: Running
Started: Fri, 03 Aug 2018 11:14:02 -0700
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/csi from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from csi-service-account-token-qgwmr (ro)
driver-registrar:
Container ID: docker://27a57b906712f5b2be9d700d54fc614be0677ace085bb12da7bc3d2f55909dfc
Image: quay.io/k8scsi/driver-registrar:v0.2.0
Image ID: docker-pullable://quay.io/k8scsi/driver-registrar@sha256:9a84ec490b5ff5390b12be21acf707273781cd0911cc597712a254bc1862f220
Port: <none>
Host Port: <none>
Args:
--v=5
--csi-address=/csi/csi.sock
State: Running
Started: Fri, 03 Aug 2018 11:14:04 -0700
Ready: True
Restart Count: 0
Environment:
KUBE_NODE_NAME: (v1:spec.nodeName)
Mounts:
/csi from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from csi-service-account-token-qgwmr (ro)
external-attacher:
Container ID: docker://88c9dbd33133c01c15b1b59dd32c7ea8104bc0862eedb30dcc43224e9804327c
Image: quay.io/k8scsi/csi-attacher:v0.2.0
Image ID: docker-pullable://quay.io/k8scsi/csi-attacher@sha256:5cbb7934bd86d400c221379cff8b24ed4c06e121ea59608cfd7e67690100ba54
Port: <none>
Host Port: <none>
Args:
--v=5
--csi-address=$(ADDRESS)
State: Running
Started: Fri, 03 Aug 2018 11:14:07 -0700
Ready: True
Restart Count: 0
Environment:
ADDRESS: /csi/csi.sock
Mounts:
/csi from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from csi-service-account-token-qgwmr (ro)
hostpath-driver:
Container ID: docker://14923bdcda87b52deae5fccf045788a17b45cc1f2bb1e95e39ead96ca6e5d26f
Image: quay.io/k8scsi/hostpathplugin:v0.2.0
Image ID: docker-pullable://quay.io/k8scsi/hostpathplugin@sha256:6c640a9b6a87e9f7261ff73be2e000367aa21f8f0c6ebfda97d4eefa5523ab53
Port: <none>
Host Port: <none>
Args:
--v=5
--endpoint=$(CSI_ENDPOINT)
--nodeid=$(KUBE_NODE_NAME)
State: Waiting
Reason: RunContainerError
Last State: Terminated
Reason: ContainerCannotRun
Message: OCI runtime create failed: open /var/run/docker/runtime-runc/moby/14923bdcda87b52deae5fccf045788a17b45cc1f2bb1e95e39ead96ca6e5d26f/state.json: no such file or directory: unknown
Exit Code: 128
Started: Fri, 03 Aug 2018 11:14:08 -0700
Finished: Fri, 03 Aug 2018 11:14:08 -0700
Ready: False
Restart Count: 0
Environment:
CSI_ENDPOINT: unix:///csi/csi.sock
KUBE_NODE_NAME: (v1:spec.nodeName)
Mounts:
/csi from socket-dir (rw)
/var/lib/kubelet/pods from mountpoint-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from csi-service-account-token-qgwmr (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
socket-dir:
Type: HostPath (bare host directory volume)
Path: /var/lib/kubelet/plugins/csi-hostpath
HostPathType: DirectoryOrCreate
mountpoint-dir:
Type: HostPath (bare host directory volume)
Path: /var/lib/kubelet/pods
HostPathType: DirectoryOrCreate
csi-service-account-token-qgwmr:
Type: Secret (a volume populated by a Secret)
SecretName: csi-service-account-token-qgwmr
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 13s default-scheduler Successfully assigned csi-pod to minikube
Normal SuccessfulMountVolume 12s kubelet, minikube MountVolume.SetUp succeeded for volume "mountpoint-dir"
Normal SuccessfulMountVolume 12s kubelet, minikube MountVolume.SetUp succeeded for volume "socket-dir"
Normal SuccessfulMountVolume 12s kubelet, minikube MountVolume.SetUp succeeded for volume "csi-service-account-token-qgwmr"
Normal Pulling 12s kubelet, minikube pulling image "quay.io/k8scsi/csi-provisioner:v0.2.1"
Normal Created 9s kubelet, minikube Created container
Normal Pulled 9s kubelet, minikube Successfully pulled image "quay.io/k8scsi/csi-provisioner:v0.2.1"
Normal Started 9s kubelet, minikube Started container
Normal Pulling 9s kubelet, minikube pulling image "quay.io/k8scsi/driver-registrar:v0.2.0"
Normal Pulling 7s kubelet, minikube pulling image "quay.io/k8scsi/csi-attacher:v0.2.0"
Normal Pulled 7s kubelet, minikube Successfully pulled image "quay.io/k8scsi/driver-registrar:v0.2.0"
Normal Started 7s kubelet, minikube Started container
Normal Created 7s kubelet, minikube Created container
Normal Pulled 5s kubelet, minikube Successfully pulled image "quay.io/k8scsi/csi-attacher:v0.2.0"
Normal Created 5s kubelet, minikube Created container
Normal Started 4s kubelet, minikube Started container
Normal Pulling 4s kubelet, minikube pulling image "quay.io/k8scsi/hostpathplugin:v0.2.0"
Normal Pulled 3s kubelet, minikube Successfully pulled image "quay.io/k8scsi/hostpathplugin:v0.2.0"
Normal Created 3s kubelet, minikube Created container
Warning Failed 3s kubelet, minikube Error: failed to start container "hostpath-driver": Error response from daemon: OCI runtime create failed: open /var/run/docker/runtime-runc/moby/hostpath-driver/state.json: no such file or directory: unknown
I have tried to create the folder manually before deploying to no avail. I'm not sure if there's a problem with my config or if it's a problem with the driver itself. Either way, the example is not working and should probably be updated to work. I'll continue working on fixing it (any help would be appreciated) and if I find a solution I'll submit a PR.
/meow
https://kubernetes-csi.github.io/docs/volume-cloning.html
Volume Cloning for Kubernetes went GA in Kubernetes 1.18. The docs need to be updated to reflect that.
The docs currently have example StorageClasses for using only fixed secret references, or only templated secret references.
It might be useful to also add an example StorageClass that combines both fixed and templated references.
And also describe some more sample user stories that this can support.
/help
The external-provisioner currently supports special key for specifying secret during volume provisioning. However, the https://kubernetes-csi.github.io/docs/secrets-and-credentials.html page does not document that.
So we should:
external-provisioner
and one for external-snapshotter
to document the "special keys" supported by both CSI volume and snapshot provisioning.@oleksiys maybe a good candidate to help with 2.
With the CSI function GA, it is necessary to update the project & sidecar container status description.
https://github.com/kubernetes-csi/docs/blob/master/book/src/Home.md
Broken link in the documentation about "enabling csidriver on kubernetes" section in k8s csi documentation -> https://kubernetes-csi.github.io/docs/csi-driver-object.html#enabling-csidriver-on-kubernetes
The current link point to https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/storage-crds while on the master there is no storage-crds.
Depending on how the sidecar containers are deployed (for example, with or without leadership election), different RBAC definitions are needed. The example in this repo contain definitions that work for this particular example, but more documentation is needed.
The example also could be improved:
ClusterRole
is too broad, it should be limited to just endpoints in the driver's namespace via a Role
(as in https://github.com/kubernetes/kubernetes/blob/60f736ed3d617f4770dc76657dd75a31bffb03f0/test/e2e/testing-manifests/storage-csi/controller-role.yaml)Two options:
docs
repo as the authoritative location for the deployment information of all sidecar containersA declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.