PVC Fail "existing disk format of " about csi-driver HOT 7 OPEN

[core@bastion storage]$ kubectl logs -f hcloud-csi-node-jcsdp -n kube-system -c hcloud-csi-driver level=info ts=2024-03-15T13:33:11.303902051Z msg="Fetched data from metadata service" id=41964065 location=nbg1 ^C [core@bastion storage]$ kubectl logs -f hcloud-csi-node-p6jj2 -n kube-system -c hcloud-csi-driver level=info ts=2024-03-15T13:33:09.161029164Z msg="Fetched data from metadata service" id=41963957 location=nbg1 level=error ts=2024-03-15T13:34:49.615777847Z component=grpc-server msg="handler failed" err="rpc error: code = Internal desc = failed to publish volume: unable to detect existing disk format of /dev/disk/by-id/scsi-0HC_Volume_100455302: disk /dev/disk/by-id/scsi-0HC_Volume_100455302 propably contains partitions"

from csi-driver.

I'm also running into this issue.

My manifest:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: task-pv-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: task-pv-pod
spec:
  volumes:
    - name: task-pv-storage
      persistentVolumeClaim:
        claimName: task-pv-claim
  containers:
    - name: task-pv-container
      image: nginx
      ports:
        - containerPort: 80
          name: "http-server"
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: task-pv-storage

The error:

MountVolume.SetUp failed for volume "pvc-15b2696c-eaf0-4e79-97de-71903a597ebb" : rpc error: code = Internal desc = failed to publish volume: unable to detect existing disk format of /dev/disk/by-id/scsi-0HC_Volume_100612315: disk /dev/disk/by-id/scsi-0HC_Volume_100612315 propably contains partitions

from csi-driver.

I am unable to reproduce this with our dev setup and the Getting Started guide.

Some questions that might help to pinpoint the issue:

• What version of csi-driver are you using?
• How often does this issue happen?
• What OS are you using on the nodes where this happens?
• What Kubernetes distribution and what version are you using?
• Did you make any modifications to the StorageClass?

from csi-driver.

• Latest from helm
• Every time
• Talos cluster
• v1.27.4 vanilla
• No

from csi-driver.

I am still unable to reproduce this. What I have done:

1. Setup a Cluster with Talos 1.7.0 using their docs
   I used Packer to create the snapshots. To generate the Talos Config I used this command instead, to get below the 32kb limit on userdata:

talosctl gen config talos-k8s-hcloud-tutorial https://$LOAD_BALANCER_IP:6443 --kubernetes-version 1.27.4 --with-docs=false --with-examples=false

2. Followed the Steps from our Getting Started on Kubernetes guide, installing the Helm chart and applying my-csi-app.

And my-csi-app successfully started and the volume is mounted.

Do you have some steps for me to reproduce this?

from csi-driver.

It looks like I get the error from the API, or it bounds the volumes wrong. I only have the problem with an older cluster. If I create a new cluster (even on old talos like 1.6), it works.

Is there some API in the client ore how the volumes that are attached are different ?
Have 2 cluster now on the same project work on one but not on the other ?

I have given up and move

from csi-driver.

Same issue

• cluster setup with: https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner (3 controller, 3 nodes)
• installed csi_driver by helm (latest)
• any hcloud volumes that i try to create fails with failed to publish volume: │ │ unable to detect existing disk format of /dev/disk/by-id/scsi-0HC_Volume_xxxxxxxx: disk /dev/disk/by-id/scsi-0 │ │ HC_Volume_xxxxxxxx propably contains partitions

from csi-driver.