snapshot stucks and PVC turns readOnlny about piraeus HOT 5 CLOSED

Sorry for the late response.

If you still see this issue: could you include the kernel log from the node that is stopping progress. See the lines:

Declined by peer node30 (id: 0), see the kernel log there

So hopefully node30 holds some clues. By the way, which version of piraeus + drbd are you using?

from piraeus.

node30-kern.log

Hi @WanzenBug!

Uploaded the full kernel-log from that timeframe.

we use the latest version of piraeus with drbd version 9.0.27-1

from piraeus.

Thanks! Looks like a process is keeping the device open, either pvdisplay or blkid.

Is there perhaps some monitoring agent running on node30 that uses these commands? If you are running some LVM commands directly on the host (like pvdisplay), you need to exclude all DRBD devices from the device scan. You can do that in /etc/lvm/lvm.conf:

devices {
    ...
    global_filter = [ "r|^/dev/drbd|" ]
    ...
}

Otherwise LVM can deadlock (as seen in the task blkid:25334 blocked for more than 120 seconds message).

from piraeus.

We are using Zabbix and monitoring the mounts for free space, this could be the reason for locking.
I will check how Zabbix-Agent gets the information

from piraeus.

We didnt had much time to test it in different ways but needed a stable solution and solved it in an other way.
Closing this issue

from piraeus.