Update from piraeus-server 1.11.1 to 1.12.3 failed with Database initialization error about piraeus HOT 2 CLOSED

Thanks for the error report.

Could I redirect you to the linstor-server project? If there is a migration issues it's almost certainly not piraeus specific.

They may ask you for a dump of the etcd keyspace to pin-point the issue. You can get that by running:

kubectl exec piraeus-op-etcd-0 -- etcdctl get --prefix /LINSTOR

from piraeus.

Dear @WanzenBug,

Thank you for your reply. The dump actually gave me a hint on what was happening. When I deleted satellites trying to fix the last problem, many snapshots stayed in the database referencing these lost nodes. I guess this happened because everything wasn't working well when I deleted the nodes. Also, it made it obvious that linstor snapshot list wasn't working either in v1.11, giving a key error for 'node'.

What I did was to process the keyspace dump, filtering out the offending snapshots, and then I issued linstor snapshot delete commands for each of them. Sorry for the inconvenience and thanks again for all the help.

from piraeus.